The Organization is a leading provider of Location data offering Global datasets to the business world by creating custom datasets based on needs. It also provides different information services, including a complete list of GIS data, Historical industry-specific data sets for Real Estate datasets for continual access, Consumer durable & goods Store location data, etc.
KEY BUSINESS CHALLENGES
The Organization was lagging in the capabilities to source the store locator’s details from multiple data sources with varying degrees of difficulty, geography, and data attributes. The critical challenge of the business was, as mentioned below,
- No central system of execution for data collection from various sources as the Client used multiple ready-to-use Scripts to scrape data from various websites.
- Unable to maintain the Scripts if the structure of any websites gets changed (Script ceases to function)
- Major Data quality issues were due to ready-to-use scripts not per the structure of the targeted websites.
- Unavailability of QA process to ensure correctness & completeness of location data
- Time-consuming data collection process to compile the data set from sources
- Poor management of all the data sources along with preparing them as per user need
- High Overhead Cost in terms of resource allocation
- Error-prone process due to multiple Human touches
SOLUTION THAT X-BYTE HAS DEVELOPED
At X-Byte, we developed a robust system to monitor more than 5000 store locators across the globe and deliver their results monthly, which can be automatically uploaded on their analytics tool.
Salient Features of the solutions
- Centralized crawling platform having inbuilt rules for Uniform Naming structure, descriptive column schema, automated quality check, and daily email alerts
- Extraction of around 5000+ global store locators data
- Keeping an eye on each newly added listing every month
- Flexible system to cope with changes in format request in less time to save cost and efforts
- Automated QA alerts with report per data point value for tracking each movement
- Allocation of skilled and dedicated team to monitor & maintain the entire system to ensure smooth functionality
Setting up the Crawler – The crawler was initially configured such that it could automatically scrape product price and essential data fields for present categories on a daily basis.
Data Template : A template was created utilizing data structuring based on the schema provided by the customer.
Delivery of Data : Without any manual input from either side, the closing data was supplied in an XML format through Data API regularly.
The dataset had all the information including comments, news timelines, most viewed articles, customer behaviour, etc. All of the scraped data was indexed using hosted indexing components, and search APIs were made available so that a client could get the results every few minutes.
The one-stop solution provided by X-Byte helped the business entity to integrate the feeds directly into their existing system for a different level of Analysis. The significant aspects that have optimized their businesses are as mentioned below,
- Automated System to extract the data monthly from more than 5000 data sources
- Automated QA processes that have minimized the time to ensure data accuracy
- Increase in customer satisfaction due to consistent solution capable of getting the location data from multiple sources in a quick time
- Allow management to focus more on essential business functions, i.e., Sales & Operational tasks.