Organization is the leading underwriting decisioning platform for commercial P&C insurers that enables the submission of documents and automatically answer underwriting questions using machine learning.


The organization was collecting data manually from various online sources which was incurring a very high cost. The traditional data collection approach was taking several days or even weeks with bottleneck of human errors, which was hampering the overall productivity. Other challnege includes unified structure of data across all the sources.



X-Byte had provided desired consultancy to scrape data from 5000 websites. Taken collaborative approach in developing Scraping Script, scheduling the scraping job, Integration of data with existing system, maintenance of software scripts, QA, bug fixes, Server & infrastructure management and 24X7 support.


Setting up the Crawler The crawler was initially configured such that it could automatically scrape product price and essential data fields for present categories on a daily basis.

Data Template : A template was created utilizing data structuring based on the schema provided by the customer.

Delivery of Data : Without any manual input from either side, the closing data was supplied in an XML format through Data API regularly.

The dataset had all the information including comments, news timelines, most viewed articles, customer behaviour, etc. All of the scraped data was indexed using hosted indexing components, and search APIs were made available so that a client could get the results every few minutes.


  •  Regular delivery of accurate, cleansed and timely data
  • Minilmied the Risk by getting the consultation from expert team
  • Enable to collect billions of data points across all the states of USA
  • Allow client to Focus more on core business due to collaborative approach
  • Enhancement in productivity with coverage from 5000+ online sources
  •  Reduce in overall cost of operation