iCapital
Create a free account
One signup unlocks everything for this job:
- Apply to this job and track your application
- See your personalized profile match score
- Generate an AI cover letter for this role
Information Technology Support Services · 201 to 500 Employees · Abu Dhabi, United Arab Emirates
Hiring: Engineering
What employees say
+ great work/life balance, good work is appreciated and recognized, positive work environment, considerate co-workers and management, many perks provided through PTO, compensation, r
− I have no cons - this is the best company I have worked for.
More reviews on the company profile.
Job description
Overview
iCapital is looking to hire an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS.
Responsibilities
- Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session or token-based login flows.
- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
- Integrate and manage complex workflows across multiple portals, APIs, and data sources.
- Contribute to architectural decisions, tooling, and best practices.
- Responsible for Code reviews and deployments.
Qualifications
- Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent professional experience
- 10+ years of experience in Python development, with a strong focus on automation and data extraction
- Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests
- In-depth experience handling multi-step authentication flows, including multi-factor authentication (MFA), CAPTCHA solving, and session or cookie management
- Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager
- Solid understanding of web protocols (i.e. HTTP, HTTPS, cookies, headers), and the ability to reverse-engineer network calls and authentication mechanisms
- Comfortable working with APIs, JSON/XML, and data transformation
- Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions
- Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal)
- Background in data engineering or ETL workflows is a plus
- Exposure to Python testing frameworks and writing unit or integration tests
Benefits
- iCapital offers a comprehensive benefits package that includes a total compensation program consisting of competitive salary, annual performance bonus, and equity for all full-time employees; healthcare with 100% employer-paid health and dental insurance; and generous paid time off (PTO).
- For additional information on iCapital Network, please visit https://www.icapitalnetwork.com/about-us Twitter: @icapitalnetwork | LinkedIn: https://www.linkedin.com/company/icapital-network-inc
Verify JavaScript for this role
Free PRACTICE for JavaScript adds a verified skill signal that can boost how roles like this rank for you — optional, not required to browse or apply.
Sign up — verify JavaScript free →Want match scores and deeper insight? Create a free candidate profile