FAQS
Can you provide the entire scraping environment for us?
Yes, we do. For the projects requiring web data aggregations from hundreds or sites and having regular data scraping frequencies, you can hire dedicated scraper team. A dedicated crawling team will take care of full end-to-end crawling operations like: scraping code development, data extraction monitoring, maintenance, quality assurance, data deliveries, communication and change requests. The team works like a part of your team!
Can you scrap information behind a login web page?
If you have an account or registered on website, you can provide our team with the credentials in order to scrap information shown behind a login. After the login, the scraping works precisely much like that of a typical scraper. Please remember, sometimes there could be limitations on number of pages to be scrapped per login.
Do I need to set up anything on my web servers or anything else?
No. Our entire web scraping scripts runs within our cloud platform. Our skilled group of scraping professionals take care of all end to end crawling operations and the customers do only receive the data output files, which in turn is a hassle free service! All you need to do is provide us the targeted sites and also we’ll scrape the listings.
If you wish our team to develop the python scraping scripts, please visit “Python Script Development” page.
Do you sign Non-disclosure Agreement (NDA)?
Yes, we do sign Non-disclosure Agreement (NDA). Because we believe in maintaining privacy and security for our clients. We respect their personal information and thus we do not share any details.
How are you unique from your rivals?
We’re a completely end-to-end web scraping company with a strong belief that a client should not run any type of scrapers or compose code to scrape the information. We have among the most effective infrastructure as well as the client driven procedure to ascertain that the task runs efficiently. We mainly focus on working for clients 24x7x365 days without even having a break. Data Quality plays a major role once the data is scrapped. Our team performs various levels of automated and manual verifications on order to deliver quality results.
We have 100s of websites to be scraped. Can you take care of jobs that huge?
Yes. Our crawling architecture is scalable enough to aggregate and handle the large scale web crawling from hundreds to thousands of websites, with any required data refresh frequency!
What mechanism do you apply if the website blocked the IP addresses?
Yes, a website could block the IPs, and even the Proxy IPs while scraping. The site could prompt for CAPTCHA verification as well. At X-Byte, being a web crawling service provider, we have large pools of Premium Paid Proxy IPs from multiple proxy providers. The Proxy IPs gets rotated every few seconds, every web requests to avoid any sort of IP detections. The Proxy IPs also gets refreshed every month or on demand.
What programming languages you are utilizing for web scraping?
We create personalise web scrapers in languages like PHP, C#, Python, VB .Net and VBA. Apart from this, on the demand of client, we also use the language as per their need to craw and extract data.
What we do not scrape?
We do not scrape pornography web sites as well as sites that have individual information like credit card particulars, Social Security Numbers and also high level sensitive personal information.
Can you collect information on a hourly/daily/weekly/monthly basis?
Yes. Our team can schedule the cron jobs on our scraping platform for required data refresh frequency. Once the data extraction is finished, it can automatically send via email, FTP, DropBox or to cloud storage locations.
Do you provide back-end support in terms of technical assistance and upkeep?
Yes, we are always there to help you out with back-end support. Because we know you might face technical bugs while operating by yourself. Hence, we will help you through all the phases without you getting stuck with your core activities of business.
How do you function?
We have an organised procedure in position:
- We gather the information and requirements from the clients that we are looking for.
- Whether client recommend us a website, or we do suggest websites, where we are going to extract the information.
- Our expert team evaluate the whole project and give you the estimate of budget, delivery time and other feedback which may help your business.
- Once the evaluation of the project gets complete, you are supposed to sign some formal documents.
- Our team of expert crawlers stat working on your project, extract the information from several and specified websites and perform tasks like data cleansing, high quality automated and manual checks.
- After passing the outcome from different phases of development and QA, we deliver the data to our client.
- Refine for Payment/Invoice & always been there to serve you in future.
How do you manage the payment?
We accept the payments by means of PayPal, SWIFT Bank Transfers, or Western Union.
In what data output format would you supply the end results?
Excel spread sheet, CSV, TSV (Tab Delimited), JSON, XML, SQL, and so on. If you have developed an API End-point, through which the data needs to be delivered in your platform, please discuss with support team.
Is your solution confidential, as well as personal?
We promise to keep you in safe hands, whether it’s about our relationship or work. We never let it spread on public forum. All the demands and information gathered from the customers will remain personal.
What does it cost?
The cost relies on a number of criteria like:
- Number of web sites (more websites, more discounted rate)
- Number of listings (more listings, more discounted rate)
- Depth Level, to crawl the required information
- Website’s framework, HTML/Flash/JavaScript/ Ajax/Java Applet
- Whether a web site blocks the IPs or prompts for captcha verifications.
What length of time does it require to extract the information?
It once again relies on above specified specifications.