Web Scraping Expert Needed for E-commerce Subcategory URL Extraction

We are seeking a skilled web scraping expert to help us extract subcategory URLs from various e-commerce websites. This feature will allow us to gather subcategory links, such as product category pages from e-commerce websites, to enhance our data collection and analysis capabilities.

Project Overview:

We require a solution that leverages BrightData to scrape multiple e-commerce websites and extract subcategory URLs. The scraper should visit each site, navigate through the main categories, and collect URLs for all available subcategories. The solution should be versatile and work on any e-commerce website from any country, such as asos.com, including subcategories like asos.com/men/shorts, asos.com/men/jeans, etc.

Requirements:

Use of BrightData:

Configure and deploy BrightData to scrape specified URLs.
Ensure that the scraper can handle multiple URLs efficiently.

Data Collection:

Scrape subcategory URLs from various e-commerce websites.
Ensure the solution can handle a variety of websites, not just one or two.

Collect URLs such as:

Main category: asos.com/men
Subcategories: asos.com/men/shorts, asos.com/men/jeans, etc.

Output:

Store the scraped URLs in a structured format (e.g., JSON, CSV).
Ensure the data is clean, accurate, and ready for integration with our system.

Qualifications:

Proven experience with web scraping and data extraction using BrightData.
Strong understanding of HTML, CSS, and JavaScript.
Familiarity with e-commerce websites and their structure.
Ability to handle anti-scraping mechanisms and ensure reliable data collection.
Attention to detail and commitment to delivering high-quality work.

Additional Requirement:

You must have access to a BrightData account to be able to build and test this. I have my own account but cannot share access to it. I will copy over the code and run it on my side.

How to Apply:

Please submit your proposal detailing your experience with similar projects, your approach to this task, and any relevant examples of past work. Include an estimated timeline and cost for completing this project.

We look forward to working with a dedicated professional to enhance our platform.