Big Data Engineer
ClearScale (headquartered in San Francisco, California, USA) - AWS Premier Consulting Partner has been offering a full range of professional cloud computing services for over 10 years, including IoT, Data Analytics, ML/AI, MLOps/DataOps, architecture design, DevOps automation, refactoring and cloud-native applications development, integration, migration, solving all sorts of security issues (from just a security check to preventing cyber-attacks) and 24/7 technical support using the best advanced technology.
The list of our customers is diverse: from governments (ClearScale is an official cloud partner of the State of California) and educational institutions (University of California, San Francisco) to well-known global brands (IBM, Samsung, GoPro, HP, Conde Nast, Carl Zeiss, etc.) The number of satisfied customers has been well over 850, some of which can be found on the company's website in the Case Studies section.
- We were the third company to gain a new AWS competency: Applied AI and Machine Learning Operations (MLOps). Less than 15 partners have this competency!
- We have numerous AWS competencies in Data Analytics and AI/ML
- We have obtained the status in the Database Freedom program (total less than 20 companies in)
- Proven expertise as a Managed Services Provider (total less than 16 companies in). This means that ClearScale can perform a full cycle consultancy and service: from audits, system or software development to the 24/7. You can read more about us on the company page - Managed Services.
Since the very foundation of the company, we work 100% remotely from various cities and countries. The professional reputation of our team members is of the highest value.
About the customer
Our customers range from SMB companies to Fortune-100 IT giants. You could expect getting experience working with multitude business domains, industries and technologies.
About the project(s)
- Data Analytics is building Data Lakes, Lake Houses with the variety of technologies, such as EMR, Glue, Athena, Redshift, Aurora, Neptune, ElasticSearch/OpenSearch, Snowflake etc.
- ML/AI projects are based on the scrupulous data analysis, hypothesis formulation and rigid model quality checks using the latest packaged SageMaker or custom ML models and frameworks.
- GenAI projects are based on AWS Bedrock
- We empower IoT companies with technologies like IoT Core, IoT SiteWise, LoRaWAN and extensive data analytics
Responsibilities:
- Assisting Team Lead/Tech Lead/(Application) Architects in requirements gathering and estimations
- Supervising Engineers on the code quality control
- Extensive data modelling by mapping data sources and discovering patterns in data
- Striking a balance between technology trade-offs to solve a business case
- Accounting for domain data source nuances
- Adapting technology to business priorities in dynamic environments (scope changes)
- Monitoring for anomalies in data pipelines
- Taking responsibility for enterprise value
- Exhibiting positivity and tenacity to work through failures
- Being curious and collaborative in incident response
Required skills and experience
ClearScale expects successful candidate to have most of the following qualifications and skills (not necessary all have to be presented):
- 3+ years of Python data-related development
- 3+ years of experience with SQL
- 2+ years of Hadoop or Spark
- 2+ years hands-on experience with AWS Glue(or Amazon EMR)
- 2+ years hands-on experience in orchestration frameworks (e.g., Airflow, Step Functions)
- 2+ years experience with DBT
- 1+ years of experience with IaC (Terraform, CloudFormation)
- 1+ years of experience with NoSQL DB (e.g., DynamoDB, CoachDB)
- English B2 or better (spoken and written)
Additionally, it would make us smile even more if you have:
- Experience in streaming analytics solutions (Kinesis, Kafka, Flink, Spark Streaming)
- Experience in CDC, Delta Lakes
We offer
# 1 Fair wage
- High hourly rate in USD
- Full-time, 40 hours per week contract
- Payments every 2 weeks
- Annual Rate Review
# 2 Professional Development
- Work with innovative Silicon Valley companies and traditional American companies at the cutting edge of digital transformation
- We work with the newest technologies in AWS cloud and open-source tools like Jira, Confluence, Lucidchart, Slack etc
- We operate in an honest and competitive environment and we are one of the AWS's top 10 key partners.
- The team willing to share its experience
- Paid AWS certifications: we provide training material, paid time off and examination itself
- Horizontal and vertical career growth - We keep growing and people keep growing with us
# 3 Strong Company Culture
- Everyone on the Clearscale team knows our company values, and lives by them in our day-to-day work. Our team enjoys high morale, flexibility, autonomy, trust, innovation, transparency, and diversity.