Data Engineer - Python

Job Overview

• Job Title: Data Engineer - Python

• Company: Asiawise Lanka Pvt Ltd

• Employment Type: Full-Time

• Location: Remote

• Experience Required: 2+ Years

Asiawise Lanka Pvt Ltd is seeking a skilled and driven Data Engineer - Python to join our growing team on a full-time, remote basis. In this role, you will design, develop, and maintain robust web scraping systems, data pipelines, and automation tools to extract actionable product intelligence from target websites. If you have strong proficiency in Python, web scraping frameworks, and database management, we would love to hear from you.

Key Responsibilities

• Web Scraping & Data Extraction: Design, develop, and maintain Python scripts and automated pipelines for large-scale web scraping and data collection from target websites.

• Product Data Processing: Extract critical product data points including titles, pricing, ratings, availability, and search listing positions.

• Scraping Framework Implementation: Implement advanced scraping solutions utilizing industry-standard libraries and frameworks such as BeautifulSoup, Scrapy, Selenium, and Playwright.

• Data Storage & Management: Store, structure, and manage extracted data efficiently across multiple formats and databases (e.g., CSV, JSON, SQL, NoSQL).

• Dashboards & Reporting: Create insightful reports and dashboards to visualize product performance, market positioning, and metrics.

• Maintenance & Adaptability: Monitor website structural changes and adapt scrapers to ensure long-term data collection continuity.

• Ethical Compliance: Ensure all data collection and scraping activities strictly comply with legal, ethical, and website guidelines.

Required Qualifications & Skills

• Education: Bachelor’s degree in Computer Science, Software Engineering, IT, or a related technical field.

• Experience: Minimum of 2 years of proven professional experience in a similar Data Engineer or Python Developer role.

• Python Expertise: Advanced proficiency in Python, with a strong focus on data extraction, web scraping, and task automation.

• Framework Proficiency: Hands-on experience with web scraping frameworks and libraries (Scrapy, BeautifulSoup, Selenium).

• API Handling: Experience working with REST APIs, authentication mechanisms, OAuth, and token management.

• AI & Autonomous Systems: Practical experience designing and implementing AI agents or autonomous systems using Python and related frameworks.

• Database & Storage Knowledge: Solid understanding of data storage systems such as MySQL, PostgreSQL, and MongoDB.

• Code Quality: Ability to write clean, scalable, maintainable, and well-documented code.

• Bonus Skills: Basic understanding of Search Engine Optimization (SEO) and product ranking mechanisms is a strong plus.

Preferred Qualifications

• Prior hands-on experience working with major e-commerce platforms such as Amazon, eBay, or Shopify.

Compensation & Benefits

• Salary: Competitive salary, negotiable based on prior experience and technical qualifications.

• Work Arrangement: 100% Remote flexibility.