Role: Azure Data Engineer Architect Location: Brentwood, TN
Description Lead Azure Data Engineer Architect role is to strategically design, develop and implement Azure Cloud based Data Warehouse, Data Lake, Data Lakehouse, Data Marts & BI solution using modern data engineering techniques and advanced analytics methods to provide information & insights to end users. The role requires modeling, collecting, harmonizing, aggregating, storing and reconciling data from variety of data sources for our petroleum refineries IT/OT technology platform including time series Database, SCADA data, batch, real time, enterprise applications, Rest API's , streaming & IOT data. The role will help design and build data pipelines, streams, data warehouse, data lake, reporting application, data generators and a whole range of tools to provide information and insight.
The ideal candidate will be responsible for designing, developing, and maintaining scalable data processing pipelines in a cloud-based environment. The candidate should have a deep understanding of Azure ADF, Databricks, streaming analytics, data engineering best practices, and a proven track record of delivering high-quality real time solutions.
Minimum of 12 to 15 years of Data Engineering, Data Warehouse/ Business intelligence with hands-on experience leading and delivering multiple full life cycle implementations. Comprehensive data warehouse analysis and design experience, with full knowledge of data warehouse methodologies and data modeling. Hands-on experience & deep understanding in designing, development and implementation of Azure Data & Analytics architecture, data modeling, transformations, and development. In-depth knowledge and minimum 5 to 7 years' experience developing & implementing data engineering solutions using Azure services for Data Warehouse, Data Lakehouse, & Advance analytics- Primarily- Azure Data Factory, ADLS, Azure Databricks, Azure Functions, Logic Apps, IoT Hub, Event Hub & Azure SQL Experience with Data pipeline and workflow management tools & Azure DevOps processes. Strong proficiency in programming languages such as Python, PySpark, or Scala End to End Implementations expertise in Azure Data Warehouse & Data Lakehouse Solid understanding of database structure principles, Master and Reference Data management Experience & expertise in optimizing the ETL/ELT data pipeline technology options to minimize the on-going operational cost of Azure data engineering & storage solutions. 8+ years experience in agile Data warehouse development methodology. 8+ years' experience designing and developing dimensional models for data warehousing 3+ years' experience in MSBI suite primarily SSIS 3+ years of implementing operational data model is plus not essential 3+ years of experience with big data batch/streaming processing. 5+ years' experience using Pyspark, Python or Scala Experience delivering reporting solution using visualization tools like PowerBI Experience in implementing SAP data analytics solution in Azure is plus not essential Experience using data modeling tool like ERWin Ability to lead and work with remote & offshore teams. Experience integrating & analyzing data from time-series databases, SCADA data, IOT, voice to text, images & videos for creating Industrial Intelligence solution. Knowledge/expertise on Azure ML/AI predictive modeling capabilities is a plus Experience using proven methods to solve business problems using Azure Data and Analytics services in combination with building data pipelines, data streams and system integration Hands-on experience with analytics and big data technologies within Microsoft Azure Cloud platform in tools such as Azure Machine Learning & Azure Cognitive Services is plus not essential Experience with data profiling, cataloguing, and data mapping for technical design using a use case-based approach that drives the construction of technical data flows Experience with Azure Administration & Security (Connection setup, User setup, Object security) is plus Understanding of statistical and machine learning modeling with experience applying these modeling techniques to business problems is a plus not essential. Ability to set up data and experimental platforms using Azure cloud service Ability to work independently with business stakeholders to collect and understand requirements for reporting & analytics. Full project management and software development life cycle experience. Experience with data process flowcharting techniques. Oil & Gas industry experience preferred but not essential. Experience & knowledge of SAP S4 and SAP BW4HANA is a plus not essential. Excellent quantitative, analytical & mathematical skills is a plus Strong understanding of the organization's goals and objectives Formal Education & Certification Bachelor's degree in computer science, information systems, or a related field, or equivalent work experience. Certifications in Azure Data Engineering and Analytics space is a plus Report this job Dice Id: ******** Position Id: ********
