[nom]

Hadoop/Databricks Administrator

Mobile: [téléphone] · Email-Id: [e-mail]

OBJECTIVE

To leverage my extensive experience in Databricks and Big Data platform administration to design secure, scalable, and cost-efficient cloud data environments, while contributing to enterprise data modernization, governance, and performance optimization initiatives.

PROFILE SUMMARY

• Having 11.8 years of Professional Experience in the field of Information Technology.
• Having 6+ years of experience in Databricks Administrator, AWS/Azure Cloud Support Engineer.
• Having 6.2 years of hands-on experience as Hadoop Administrator in Cloudera, HDP & MapR Distribution.
• Hands on experience in ecosystem components Hive, HBase, Sqoop, Oozie, Zookeeper, Ranger, Kerberos, Spark, Hue, CI/CD, GitLab, AWS, Azure, Linux, Shell Script, Ansible, Kubernetes, Unity Catalog, Terraform, RBAC.
• Hands-on experience in administration of AWS Databricks and Azure Databricks environments across Development, UAT and Production environments.
• Strong experience in Databricks Workspace Administration, Cluster Administration, Security Governance and Platform Operations.
• Experience in implementing Databricks Lakehouse Architecture using Delta Lake and Unity Catalog.
• Hands-on experience in managing Databricks Workspaces, Clusters, Cluster Policies, Notebooks, Jobs and Workflows.
• Experience in creating and managing Job Clusters and All-Purpose Clusters based on workload requirements.
• Experience in configuring Cluster Policies to enforce governance, security standards and cost optimization.
• Hands-on experience in Databricks Runtime (DBR) upgrades, compatibility testing and platform patching activities.
• Experience in implementing and managing Unity Catalog for centralized governance of data assets.
• Experience in managing Catalogs, Schemas, External Locations, Storage Credentials and Volumes using Unity Catalog.
• Implemented fine-grained access control using Unity Catalog, Azure Active Directory Groups, and IAM Roles.
• Experience in integrating Databricks with Azure Active Directory (AAD) and AWS IAM for authentication and authorization.
• Configured VNet Injection, Secure Cluster Connectivity (SCC) and Private Endpoints for secure access.
• Configured NSGs, VNets, Route Tables and subnet-level controls for workspace security.
• Integrated Azure Active Directory for Single Sign-On (SSO) and SCIM-based User and Group Provisioning.
• Experience in onboarding users, managing groups, workspace permissions, and access governance.
• Configured and managed Databricks Workspaces, Clusters and Notebooks on AWS and Azure cloud platforms.
• Experience in administering Delta Lake tables and performing OPTIMIZE, VACUUM and storage maintenance activities.
• Good understanding of Medallion Architecture, Delta Lake, Delta Live Tables (DLT) and Databricks Workflows.
• Experience in managing Databricks Jobs, Scheduling, Monitoring and Alerting for enterprise workloads.
• Hands-on experience in automating Databricks Workspace and Cluster deployment using Terraform and Infrastructure as Code (IaC).
• Created S3 buckets and S3 life cycle management for old data and logs. Also adding EBS and EFS whenever required.
• Configured and managed Databricks workspaces, clusters, and notebooks on cloud platforms (AWS).
• Proficient knowledge of the Databricks Platform, including experience with Databricks Workspaces, Apache Spark, Delta Lake and Medallion Architecture, Unity Catalog, Delta live tables and Workflows.
• Managed user roles, permissions, and access control using IAM, RBAC, and Databricks' security features.
• Experience in automating Databricks administration activities using REST APIs, Terraform, Ansible and Shell Scripting.
• Implemented IAM Roles, Instance Profiles and Cross-Account Access for secure access to AWS resources from Databricks.
• Managed Databricks clusters on AWS using EC2 instances with auto-scaling and auto-termination configurations.
• Experience configuring Databricks access to Amazon S3, AWS Glue Catalog and AWS Secrets Manager.
• Integrated Databricks with AWS IAM for authentication and role-based access control.
• Experience in monitoring Databricks environments using Azure Monitor, CloudWatch, Grafana and native Databricks monitoring tools.
• Managed secrets using Azure Key Vault integrated with Databricks.
• Experience configuring IP Access Lists and Workspace Security Controls.
• End to End Implementation HDP, CDP (Dev/PROD).
• Hands on experience in installation, configuration, supporting and managing Hadoop Clusters.
• Decommissioning and commissioning the Node on running Hadoop cluster.
• Hands on experience on data transfer/migration across the clusters from HDP to CDP.
• Expertise in upgrading/rolling upgrading in Cloudera Product like CDP, HDP.
• Enabling High Availability (HA) for various components in Hadoop Cluster Production Environment.
• Rebalancing the Hadoop Cluster.
• Experience in Kafka application tuning for real-time warehousing applications.
• Hands on experience with Hadoop Security in Ranger and Kerberos.
• Troubleshooting, diagnosing, performance tuning, and solving the Hadoop issues.
• Provided L2/L3 production support for Databricks platform incidents and critical business applications.
• Experience in Root Cause Analysis (RCA) and Problem Management for recurring platform issues.

PROFESSIONAL SUMMARY

• Working with BNP Paribas, Bangalore as Technical Lead from June 2024 to Till Now.
• Worked with LTI Mindtree, Bangalore as Specialist – System Engineering from Sept 2021 to June 2024.
• Worked with TCS Pvt Ltd, Bangalore as IT Analyst from May 2019 to Sept 2021.
• Worked with HCL infosystems Ltd, Bangalore, as Senior System Engineer from June 2018 to April 2019.
• Worked with Baryons Software Solutions Pvt Ltd, Bangalore as Software Engineer from Oct 2014 to May 2018

TECHNICAL EXPERTISE

• Operating Systems: Centos And RHEL
• Relational DBMS: MySQL, Oracle Database.
• Big Data & Lakehouse Platform: Databricks, Cloudera CDP/HDP, MapR
• Cloud Platform:
• AWS: EC2, S3, IAM, VPC, ELB, EBS, RDS, CloudWatch, Lambda, ECS
• Azure: Azure Databricks, Azure Data Lake Storage Gen2 (ADLS), Azure Active Directory (AAD), Azure Key Vault, VNet Injection, Private Endpoints, Azure Monitor
• Hadoop Ecosystem: Hadoop (HDFS, YARN, MapReduce), Hive, HBase, Spark, Kafka, Oozie, Sqoop, Zookeeper, NiFi, Cloudera CDP, HDP, MapR.
• Security & Governance: Kerberos, Ranger, RBAC, IAM, Unity Catalog Privileges, Data Encryption, Secrets Management.
• Scripting & Automation: Shell Script, Ansible, Terraform
• Monitoring & Ticketing Tools: Service Now, JIRA, HPSM, Grafana, Dynatrace, ELK, CloudWatch, Azure Monitor
• Change Management Tool: HPSM, SharePoint
• Reporting tools: Pentaho BI Tool

EDUCATION SUMMARY

• B.E (Pass out 2013) from RGTU Bhopal.

PROFESSIONAL EXPERIENCE

Project 1: · BNP Paribas

Organization: · BNP Paribas India Solutions Pvt. Ltd.

Technologies: · AWS Databricks, Azure Databricks, Unity Catalog, Delta Lake, Spark, Terraform,

Ansible, CDP, HDP, Hive, HBase, Ranger, Kerberos, ADLS Gen2, S3, CloudWatch, Azure

Monitor.

Responsibilities:

• Administering enterprise-scale AWS Databricks and Azure Databricks platforms across Development, UAT and Production environments.
• Managing Databricks workspaces, clusters, cluster policies, notebooks, jobs and workflows.
• Implemented Unity Catalog for centralized governance, data access management, storage credentials, and external locations.
• Managed catalogs, schemas, tables, volumes and external locations using Unity Catalog.
• Integrated Databricks with Azure Active Directory (AAD) and AWS IAM for secure authentication and authorization.
• Configured SCIM provisioning for automated user and group synchronization.
• Implemented Role-Based Access Control (RBAC) and fine-grained data access controls using Unity Catalog.
• Configured VNet Injection, Secure Cluster Connectivity (SCC), Private Endpoints and NSGs to secure Databricks environments.
• Managed Databricks secrets using Azure Key Vault and cloud-native security services.
• Created and enforced Cluster Policies to standardize compute resources and control cloud costs.
• Designed auto-scaling and auto-termination policies to optimize resource utilization and reduce operational expenses.
• Administered Job Clusters and All-Purpose Clusters supporting enterprise analytics and ETL workloads.
• Performed Databricks Runtime (DBR) upgrades, compatibility validation and platform maintenance activities.
• Managed Delta Lake optimization activities including OPTIMIZE, VACUUM and storage maintenance operations.
• Monitored platform health using Azure Monitor, CloudWatch, Grafana and Databricks monitoring tools.
• Developed monitoring dashboards and alerting mechanisms for cluster health, job failures and infrastructure utilization.
• Automated Databricks workspace provisioning, cluster deployment and configuration management using Terraform and Ansible.
• Managed 15+ Hadoop clusters (600+ nodes) including capacity planning and cluster expansion.
• Implemented Kerberos, Ranger, and HDFS encryption for enterprise data security.
• Led HDP to CDP migration activities including cluster validation, node commissioning/decommissioning and data migration.
• Performed Hadoop cluster administration including capacity planning, cluster expansion, rebalancing and health monitoring.
• Integrated Unity Catalog with AWS S3 for centralized data governance and access management.
• Managed Storage Credentials, External Locations and Catalog permissions using Unity Catalog.
• Configured secure connectivity between Databricks and AWS services using VPC Peering, Security Groups and Private Link.
• Managed Databricks access to AWS services including S3, RDS, ECS, Lambda and Glue Catalog.
• Configured Databricks Secret Scopes and integrated AWS Secrets Manager for secure credential management.
• Monitoring the daily system and services connectively and security on Production & Developer Cluster.
• Automated cluster creation and notebook deployments using Ansible & REST APIs.
• Supported data engineers with access, debugging, and notebook/job issues.
• Cluster expansions (adding nodes to the cluster).
• Involving in change management window activities.

Project 2: · ExxonMobil

Organization: · LTIMindTree Pvt. Ltd.

Technologies: · Azure Databricks, AWS, CDP, HDP, ADLS Gen2, Hive, HBase, Spark, Ranger, Kerberos.

Responsibilities:

• Administered Azure Databricks and AWS-based Databricks clusters supporting data engineering and analytics workloads.
• Optimized cluster configurations, auto-scaling, and instance selection for performance and cost efficiency.
• Integrated Databricks with Azure Data Lake Storage Gen2 for secure and scalable storage.
• Managed access control using Azure Active Directory, IAM roles, Ranger policies, and Unity Catalog governance.
• Automated Databricks workspace deployment, cluster provisioning and policy management using Terraform.
• Developed reusable Terraform modules for Databricks infrastructure deployment on AWS.
• Performed Databricks Runtime (DBR) upgrades and cluster patching activities across AWS environments.
• Managed Databricks Jobs and Workflows for large-scale ETL and data engineering pipelines.
• Automated cluster lifecycle management, job scheduling, and provisioning using scripts and automation tools.
• Implemented monitoring dashboards for platform health, job tracking, and resource utilization.
• Provided L2/L3 production support, resolving P1/P2 incidents within SLA timelines.
• Reformed Spark performance tuning and workload optimization.
• Managed Kerberos authentication, Ranger policies, and Hadoop security configurations.
• Conducted backup, snapshot, and disaster recovery procedures for Hadoop environments.
• Collaborated with network, infrastructure, and application teams for end-to-end platform reliability.
• Managed node commissioning and decommissioning during cluster migration phases.

Project 3: · HP Inc.

Organization: · TCS Pvt Ltd

Technologies: · MapR, Hortonworks, Hive, HBase, Ranger, Sqoop, Kerberos.

Responsibilities:

• Installed Hadoop Clusters for PROD, ITG, DEV and POC in MapR and Hortonworks.
• Collaborated with multiple teams for design and implementation of Hadoop clusters.
• Responsible for commissioning & decommissioning of nodes from Clusters.
• Responsible for disk repairing in cluster.
• Rebalancing the Hadoop Cluster in Hortonworks.
• Working on Name node high availability.
• Allocating the name and space Quotas to the users in case of space problems as per the analysis in Grafana.
• Involved in Installing and configuring Kerberos for the authentication of users and Hadoop daemons.
• Implemented Kerberos integration with LDAP.
• Implementing and troubleshooting SSH key based password less authentication.
• Implemented scripts for Kerberos keytab generation.
• Configured the mirroring setup in MapR cluster for data transfer.
• Good knowledge on hive, HBase and oozie installation and DB configuration for the same.
• Installing and upgradation of Packages and patches according to the client requirement.
• Changing file permissions as per the client request.
• Good exposure in coordinating with vendor related issues for all kinds of hardware failures.
• Processes administration and Management like monitoring, start/stop/kill various process

Project 4: · UIDAI- Aadhar

Organization: · HCL infosystems Ltd

Technologies: · MAPR, Hdfs, Hive, MAPRDB, HBase, Sqoop, Pig, Control Minder

Description:

Unique identification project was initially conceived as an initiative that would provide Identification for each resident across the country and would be used primarily as the basis for efficient delivery of welfare services.

Responsibilities:

• Cluster maintenance, commissioning, and decommissioning data nodes.
• Installation and configuration of MapR Hadoop cluster, Design & develop MapR DR setup, and manage data on MapR cluster
• End-to-end performance tuning of MapR clusters and Hadoop Map/Reduce routines against very large data sets, working with MapR cluster along with MapR-Table (creation, import, export, scan, list)
• Managing and monitoring clusters.
• Performed data balancing on clusters
• Applications for PROD Support as Roaster and Hadoop Platform Support.
• Managing MFS cluster users (MAPR), permissions and Application users access.
• Working on Name Node with high availability customizing zookeeper services.
• Improve speed, efficiency and scalability of the continuous integration environment
• Managing quotas to MapR File System.
• Recovering from node failure and troubleshooting common Hadoop cluster issues.
• Responsible for MapR File system data rebalancing.
• Responsible for performing the backup and Restoration of data from MFS to SAN and Tapes as per Retention Policy.
• Coordinating with team members for proper resolution of tickets
• Checking daily jobs and space alerts
• Manage and review Hadoop log files.
• Troubleshooting day-to-day issues, such as login problems, network issues, and permission issues.

Project 5: · Exigent (Chameleon Post Signoff)

Organization: · Baryons Software Solutions Pvt. Ltd

Technologies: · Java, Pentaho BI, MySQL, Apache Hadoop, JIRA,

Description:

Exigent has a software product roadmap to build a Contract Management product that can be implemented both on-cloud and on premise for Exigent’s customers.

Responsibilities:

• Load the processed data from each upstream application to HDFS via SQOOP
• Experience in analyzing structured data using HIVE, PIG
• Involving in developing the Hive Reports.
• Monitoring and managing the Hadoop cluster.
• Working on setting up Hadoop multi node clusters, pig, Hive and HBase.
• Using Sqoop extensively to import data from RDMS sources into HDFS.
• Performed transformations, cleaning and filtering on imported data using Hive and loaded final data into HDFS.
• Involving in creating external tables, partitioning, and bucketing of table.
• Responsible for creation and setting up of environment and re-configuration activities
• Created reports for the BI team using Sqoop to export data into HDFS and Hive.

I hereby declare that all the information furnished above is true to the best of my knowledge.

[nom]