Sr. Principal Data Engineer (Remote/Hybrid)

We are looking for an adept, architecturally minded Senior Principal Data Engineer to lead a team designing and building out a multi-tenant data mesh to enable our soon-to-be-launched digital transformation product which uses advanced NLP, knowledge engineering, and ML to accelerate innovation in engineering, manufacturing, and scientific operations. You’ll be a pioneering Data Engineering leader and be empowered to create a strong community of Data Engineers. The perfect candidates will have strong data infrastructure and data architecture skills, a proven track record of leading and scaling data teams, strong operational skills to drive efficiency and speed, strong project leadership, and a strong vision for how data engineering can proactively create positive impact for companies. You’ll split your time between unblocking your team, vetting their designs, and coaching them to realize their full potential, while also building things yourself.

Job Responsibilities

• Design, build, and maintain a multi-tenant Data Mesh within the AWS cloud comprised of Data Lakes, Warehouses, Streaming, Graphs, and analytical NoSQL stores
• Proactively mentor and grow the data engineering team, identify the required skills, hire the best talent, actively manage their performance, and help them to grow to their fullest potential.
• Drive adoption and standardization of data governance, lineage, cataloging, and stewardship practices across teams
• Work closely with data scientists, micro-service developers, and security experts to build out a big data platform incrementally and securely
• Work closely with the product management and development teams to rapidly translate the understanding of customer data and requirements to product and solutions
• Maintain an excellent understanding of the business’s long-term goals and strategy and ensures that the design and architecture are aligned with these
• Define and manage SLA’s for data sets and processes running in production
• Design for disaster recovery balancing availability and consistency in multi-region scenarios
• Research and experiment with emerging technologies and tools related to big data
• Establish and reinforce disciplined software engineering processes and best-practices

Ideal Qualifications

• Comfort and ideally substantial experience operating big data infrastructure in a cloud-based ecosystem (AWS preferred)
• Deep understanding of the theoretical and practical tradeoffs of various data formats in object/file stores (Parquet, Avro, JSON, etc.) in combination with a variety of ETL tools (Spark, Presto, etc.)
• Deep understanding of the theoretical and practical tradeoffs of various NoSQL stores (Cassandra, Elasticsearch, DynamoDB, etc.) with respect to different read/write patterns and availability/consistency requirements
• Mastery of operating and designing stream-based data systems (Kafka, AWS Kinesis, GCP PusSub, etc.) particularly under varying load
• Be proficient in modern big data architectural approaches (Kappa/Lambda architectures, Data Lake Zones, etc.)
• Experience with data governance, lineage, cataloging tooling (Apache Atlas, Apache Ranger, AWS Glue Catalog, etc.)
• Experience with data pipeline and workflow management tools (AWS Data Pipeline, Apache Airflow, Argo, etc.)
• Experience with stream-processing systems (ksqlDB, Spark Streaming, Apache Beam/Flink, etc.)
• Experience with software engineering standard methodologies (unit testing, code reviews, design document, continuous delivery)
• Strong, effective communication skills to bring multiple personalities and cultures together to work towards a common goal.
• Ability to conceptualize and articulate ideas clearly and concisely
• Management and leadership experience with a history of initiating and driving projects successfully
• Entrepreneurial or intrapreneurial experience where you helped lead the creation of a new product & organization

Nice to Have’s

• Strong algorithms, data structures, and coding background with either Java, Python or Scala programming experience
• Experience working with knowledge graphs stores (Stardog, TigerGraph, Ontotext GraphDB, Neo4j) and surrounding semantic technology (OWL, RDF, SWRL, SPARQL, JSON-LD)
• Experience working with Snowflake data warehouses and dimensional modeling practices
• BA/BS or Masters in Computer Science, Math, Physics, or other technical fields
• Experience with at least 10+ terabyte datasets, ideally up to multiple petabytes

What We Offer

• Competitive base salary, bonus plans and equity.
• A comprehensive, benefits package that includes medical, dental, vision and life insurance plans, paid time off, a generous 401k match with no vesting period, parental leave and 3 volunteering days each year. For more information on benefits, please access the benefits page on our careers site: https://careers.ihsmarkit.com/benefits.php.
• For work locations in the state of Colorado, the anticipated minimum base salary for this role would be $138,348. Compensation will be determined by the education, experience, knowledge, and abilities of the applicant.

We’re building a software solution that connects data in revolutionary ways, illuminating answers that were previously impossible to find and empowering our clients to envision the future so they can determine the best course of action in the present. Join us!

-----------------------------------------------

Equal Opportunity Employer:

S&P Global is an equal opportunity employer and all qualified candidates will receive consideration for employment without regard to race/ethnicity, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, marital status, military veteran status, unemployment status, or any other status protected by law.  Only electronic job submissions will be considered for employment.

If you need an accommodation during the application process due to a disability, please send an email to: EEO.Compliance@spglobal.com and your request will be forwarded to the appropriate person. 


Apply