A leading Venture Capital firm in Hong Kong is looking for a data engineer to join their growing team. In this role, you will maintain data pipelines to transform raw data into useful data. It will be a combination of both data engineering and data analytics to effectively transform and process data. The ideal candidate should have data engineering skills (SQL, Python) and strong analytical skills to succeed in this data engineering role.
- Develop technical architecture to enable Analytics and Data Science using industry best practices for large scale processing;
- Work with the management and research teams to come up with effective and useful analysis based on processed data.
- Design, develop and implement ETL data pipelines for batch and streaming solutions;
- Research and develop distributed crawler and data acquisition system; optimize the crawling strategy and improve the crawling effect;
- Monitor data quality across data processing lifecycle.
- Bsc. in relevant disciplines with experience in Data Engineering/Data Science;
- Proficient with one of the following programming languages - Python, SQL, PHP, JAVA, C++
- Proficient in data pipeline development including batch or streaming processing;
- Familiar with concepts and techniques of web-scraping; Able to design and develop crawler system an advantage;
- Experience with AI frameworks (e.g Tensorflow, Keras)