Senior Data Engineer
📌 Data Development Engineer
Location: Beijing | Shanghai | Full-time | Onsite only
🧩 About the Role
We are looking for a skilled Data Development Engineer to join our data team. You will be responsible for designing, developing, and maintaining efficient data pipelines, focusing on web scraping and large-scale data acquisition to support various business needs. The ideal candidate should have strong data engineering skills and solid experience in web scraping, able to quickly and reliably collect and process multi-source heterogeneous data.
🛠️ What You'll Do
- Design, develop, and maintain stable and efficient data acquisition systems including web crawlers, API data extraction, and data cleaning
- Manage large-scale data storage, scheduling, and ensure data integrity and real-time availability
- Optimize crawler performance and address anti-scraping challenges
- Collaborate closely with data science, product, and business teams to fulfill diverse data requirements
- Monitor data pipelines, quickly troubleshoot and resolve data anomalies, ensuring data quality and system stability
✅ What We're Looking For
Technical skills:
- Proficient in Python with strong experience in web scraping frameworks such as Scrapy, Requests, Selenium, etc.
- Familiar with data processing technologies including SQL, ETL processes, and common databases (MySQL, MongoDB, Redis, etc.)
- Experience in designing and implementing distributed crawling systems and dealing with anti-crawling mechanisms
- Comfortable with Linux development and debugging, including shell scripting
- Strong understanding of data structures, algorithms, and code optimization
Soft skills:
- Excellent communication and teamwork skills, able to collaborate effectively with product and business teams
- Highly proactive and responsible, capable of problem-solving under pressure
- Strong attention to data quality, with a focus on writing clean, maintainable, and robust code
🎯 Bonus Points
- Experience in financial or tech industry data engineering or quantitative development
- Familiarity with big data ecosystems such as Spark, Flink, Kafka, etc.
- Experience in Python automation and scheduling tools
- Contributors to large-scale scraping projects or open-source scraping tools preferred
FAQs
Congratulations, we understand that taking the time to apply is a big step. When you apply, your details go directly to the consultant who is sourcing talent. Due to demand, we may not get back to all applicants that have applied. However, we always keep your resume and details on file so when we see similar roles or see skillsets that drive growth in organizations, we will always reach out to discuss opportunities.
Yes. Even if this role isn’t a perfect match, applying allows us to understand your expertise and ambitions, ensuring you're on our radar for the right opportunity when it arises.
We also work in several ways, firstly we advertise our roles available on our site, however, often due to confidentiality we may not post all. We also work with clients who are more focused on skills and understanding what is required to future-proof their business.
That's why we recommend registering your resume so you can be considered for roles that have yet to be created.
Yes, we help with resume and interview preparation. From customized support on how to optimize your resume to interview preparation and compensation negotiations, we advocate for you throughout your next career move.