[nom]

Hadoop/Cloudera Administrator

Mobile: [téléphone] · Email-Id: [e-mail]

OBJECTIVE

A dedicated Senior Level Hadoop with proven experience, with an emphasis on large secure data systems and larger environments.

PROFILE SUMMARY

• Having 11.7 years of Professional Experience in the field of Information Technology.
• Having 8.6 years of hands-on experience as Hadoop/Cloudera Administrator in Cloudera, HDP & MapR Distribution.
• Hands on experience on ecosystem components Hive, HBase, Oozie, Zookeeper, Ranger, Kerberos, Spark, Hue, Nifi, Oracle, Linux, SSH, Ansible, Shell Scripting, GitLab, ELK, CDSW, CDS, Kubernetes, Atlas, AWS.
• Hands-on experience in installing, configuring and managing Cloudera Data Platform clusters using Cloudera Manager.
• Strong experience in monitoring cluster health, services, and hosts using Cloudera Manager dashboards and alerts.
• Expertise in configuring and managing Cloudera Manager alerts, health checks, and service monitoring.
• Hands-on experience in managing Cloudera parcels for service installation and upgrades.
• Experience in Cloudera Manager backup and recovery for cluster configuration and metadata.
• Expertise in configuring HDFS High Availability (HA) using Active and Standby Name Nodes in Cloudera clusters.
• End to End Implementation HDP, CDP (Dev/PROD).
• Having good experience on AWS services like Elastic Compute Cloud (EC2), IAM, S3, ELB, RDS, VPC, Route53, Cloud watch, EBS, LAMBDA, ECS.
• Created S3 buckets and S3 life cycle management for old data and logs. Also adding EBS and EFS whenever required.
• Expertise in Cloudera Data Platform (CDP) and Cloudera Data Services (CDS) including CDW and CML components.
• Hands on experience on data transfer/migration across the clusters from HDP to CDP.
• Experience in Kafka application tuning for real time warehousing application.
• Creating, Managing and safeguarding Snapshots, Trash configuration, Backup and Disaster Recovery.
• Hands on experience on mirroring a volume in MapR.
• Hands on experience on hive and HBase data migration.
• Expertise in Cluster Installation for POC, Dev, Staging and Production environment
• Troubleshooting, diagnosing, tuning and solving the Hadoop issues.
• Worked on importing and exporting data from MySQL databases into HDFS and Hive using Sqoop
• Involved in Hive table creation, partitioning and bucketing of tables
• Experience in scripting for automating tasks. (Shell Scripting, Ansible Tool).
• Expertise in Hadoop security Authentication using MIT Kerberos & LDAP, Authorization using Ranger, and extended ACLs for securing the cluster & it’s components.
• Experience in renewal of SSL Certs/Cloudera license/Truststore and Keystore.
• Managing and troubleshooting multiple cluster issues within SLA to avoid impact on business.
• Hands-on experience in installation, configuration and administration of Kubernetes clusters (On-Prem / Cloud).
• Worked in 24x7 environment for production support in an on-call rotation.
• General Administration of Linux installation, Users Creation, Group Creation, Permissions, and package management using YUM and RPM.
• Ability to quickly master new concepts and technology.
• Experience in managing Kubernetes components such as API Server, Controller Manager, Scheduler, etcd and kubelet.
• Created and managed Kubernetes objects including Pods, Deployments, ReplicaSets, DaemonSets and Services.
• Configured and managed Namespaces, Resource Quotas, Limit Ranges and Role-Based Access Control (RBAC).
• Hands on experience in installation, configuration, supporting and managing Hadoop Clusters.
• To work on cluster maintenance which includes maintenance of other dependencies services (backend databases maintenance, Linux team maintenance).

PROFESSIONAL SUMMARY

• Working with BNP Paribas as Technical Lead from June 2024 to Till Now.
• Worked with LTIMindtree as Specialist – System Engineering from Sept 2021 to June 2024.
• Worked with TCS Pvt Ltd, Bangalore as IT Analyst from May 2019 to Sept 2021.
• Worked with HCL infosystems Ltd, Bangalore, as Senior System Engineer from June 2018 to April 2019.
• Worked with Baryons Software Solutions Pvt Ltd, Bangalore as Software Engineer from Oct 2014 to May 2018

TECHNICAL EXPERTISE

• Operating Systems: Centos And RHEL
• Hadoop Distributions: CDP, HDP & MapR
• Cloudera Data Platform: CDP, CDSW
• Cloud Platforms: Amazon Web Services (AWS).
• Hadoop Ecosystem: HDFS, Sqoop, Hive, HBase, PySpark, Oozie, Sqoop, Yarn, NiFi, Hue, Atlas, Iceberg.
• Ticketing Tools: Service Now, JIRA, HPSM
• Orchestration Tools: Kubernetes (Cluster Administration, POD, Deployment, RBAC, Namespaces, PV/PVC, Resource Quotas)
• Change Management Tool: ServiceNow, JIRA, HPSM
• Security: Kerberos, SSL, Ranger, ACL
• Automation & Scripting: Ansible, Shell Scripting
• Monitoring & Logging: Cloudera Manager, Dynatrace, ELK Stack, Grafana.
• Reporting tools: Pentaho BI Tool
• Version Control: GitLab
• Relational DBMS: MySQL, Oracle Database

EDUCATION SUMMARY

• B.E (Pass out 2013) from RGTU Bhopal.

PROFESSIONAL EXPERIENCE

Project 1: · BNP Paribas

Organization: · BNP Paribas India Solutions Pvt. Ltd.

Technologies: · HDP, CDP, CDSW, Hive, HBase, PySpark, Ranger, Ansible, Hue, Oozie Kerberos,

Kubernetes, Linux, AWS, Atlas, Iceberg.

Responsibilities:

• Managing over 15 Bigdata/Hadoop clusters and over 600 nodes with Apache Hadoop.
• Installation and managing the clusters with the Configuration management tools like Ansible.
• Installation of various Hadoop Ecosystems and Hadoop Daemons
• Installed and configured CDP Private Cloud Base clusters using Cloudera Manager including parcel deployment, repository setup, and service configuration.
• Configured Cloudera Auto-TLS and managed certificate lifecycle including keystore and truststore updates.
• Troubleshooting the failed jobs and failed services in the cluster.
• Installed CDP 7.1.3 in one of 120 node cluster.
• Performed cluster capacity planning and storage balancing using HDFS rebalancer to optimize disk utilization.
• Facilitated commissioning and decommissioning of nodes and added services on nodes in CDP cluster, enhancing cluster scalability and performance.
• Backup and Recovery task by creating Snapshot policies.
• Involving in capacity planning in view of current utilization and future requirements of Hadoop environment.
• Worked on Cloudera Data Services (CDS) components including CDW and CDE.
• Managed node provisioning, commissioning, and decommissioning activities for cluster scaling and maintenance.
• Troubleshot Hive, Spark, and YARN job failures using Cloudera Manager logs, service logs, and metrics analysis.
• Configured and managed Kubernetes namespaces, resource limits, and persistent volumes.
• Monitored pod health, node utilization, and container lifecycle using kubectl and Cloudera Manager.
• Experienced in setting up AWS VPC, including subnets.
• Manage AWS load Balancing, Autoscaling, IAM roles and policies.
• Experience in AWS S3 bucket and policies of buckets for backup.

Project 2: · ExxonMobil

Organization: · LTIMindTree Pvt. Ltd.

Technologies: · Hortonworks, CDP, hive, HBase, Ranger, Sqoop, Spark, NiFi, CDSW, Kerberos.

Responsibilities:

• Monitoring the daily system and services connectively and security on Production & Developer Cluster.
• Identify and monitor computational resources usage on the cluster for better data handling.
• Data Migration from HDP to CDP environment hence we need to coordinate with use case team to migrate data from source (HDP) to Destination (CDP).
• Managing authentication of using Kerberos.
• Working on all service requests by using service now ticketing tool – Cloudera manager on prem, Ambari UI, CDP Public, AWS Cloud.
• Managed CDSW/CML environments including workspace provisioning and runtime configuration.
• Worked with Kubernetes backend supporting CDSW platform.
• Managed user access and security integration with Kerberos.
• Assisted in model deployment and monitoring in CDSW platform.
• Data Migration from HDP to CDP environment hence we need to coordinate with use case team to migrate data from HDP to CDP
• If facing an error in any job so find the root cause and try to restart it again.
• Resolving P1, P2 incidents for production issues and resolve it as per SLA
• Coordinate with HPC team if any issues like server Hardware issue, NTP sync, server not reachable, software Repository kind of issues
• Adding new nodes to CDP production cluster & Decommissioning nodes from HDP (Old cluster) one by one as part of data migration.

Project 3: · HP Inc.

Organization: · TCS Pvt Ltd

Technologies: · MapR, Hortonworks, hive, HBase, Ranger, Sqoop, Kerberos.

Responsibilities:

• Installed Hadoop Clusters for PROD, ITG, DEV and POC in MapR and Hortonworks.
• Collaborated with multiple teams for design and implementation of Hadoop clusters.
• Responsible for commissioning & decommissioning of nodes from Clusters.
• Maintaining cluster health and HDFS space for better performance.
• Responsible for disk repairing in cluster.
• Rebalancing the Hadoop Cluster in Hortonworks.
• Working on Name node high availability.
• Allocating the name and space Quotas to the users in case of space problems as per the analysis in Grafana.
• Installation of various Hadoop Ecosystems and Hadoop Daemons.
• Involved in Installing and configuring Kerberos for the authentication of users and Hadoop daemons.
• Implemented Kerberos integration with LDAP.
• Implementing and troubleshooting SSH key based password less authentication.
• Implemented scripts for Kerberos keytab generation.
• Able to transfer the data across the cluster in Hortonworks.
• Configured the mirroring setup in MapR cluster for data transfer.
• Good knowledge on hive, HBase and oozie installation and DB configuration for the same.
• Installing and upgradation of Packages and patches according to the client requirement.
• Changing file permissions as per the client request.
• Good exposure in coordinating with vendor related issues for all kinds of hardware failures.
• Processes administration and Management like monitoring, start/stop/kill various process Project 4: · UIDAI- Aadhar Organization: · HCL infosystems Pvt. Ltd.

Technologies: · MAPR, Hdfs, Hive, MAPRDB, HBase, Sqoop, Pig, Control Minder

Description:

Unique identification project was initially conceived as an initiative that would provide

Identification for each resident across the country and would be used primarily as the basis for efficient delivery of welfare services.

Responsibilities:

• Cluster maintenance, commissioning & decommissioning data nodes.
• Installation and configuration of MapR Hadoop cluster, Design & develop MapR DR setup, and manage data on MapR cluster.
• End-to-end performance tuning of MapR clusters and Hadoop Map/Reduce routines against very large data sets, working with MapR cluster along with MapR-Table (creation, import, export, scan, list).
• Managing & monitoring cluster.
• Performed data balancing on clusters
• Applications PROD Support as roaster and Hadoop Platform Support.
• Managing MFS cluster users (MAPR), permissions and Application users' access.
• Working on Name Node high availability customizing zookeeper services.
• Improve speed, efficiency and scalability of the continuous integration environment.
• Managing quotas to MapR File System.
• Recovering from node failure and troubleshooting common Hadoop cluster issues.
• Responsible for MapR File system data rebalancing.
• Responsible for performing the backup and Restoration of data from MFS to SAN and Tapes as per Retention Policy.
• Coordinating with team members for proper resolution of tickets
• Checking daily jobs and space alerts
• Manage and review Hadoop log files.
• Troubleshooting day-to-day issues, such as login problems, network issues, permission issues.

Project 5: · Exigent (Chameleon Post Signoff)

Organization: · Baryons Software Solutions Pvt. Ltd

Technologies: · Java, Pentaho BI, MySQL, Apache Hadoop, JIRA, SVN

Description:

Exigent has a software product roadmap to build a Contract Management product that can be implemented both

On-cloud and on premise for Exigent’s customers.

Responsibilities:

• Load the processed data from each upstream application to HDFS via SQOOP.
• Experience in analyzing structured data using HIVE, PIG
• Involving in developing the Hive Reports.
• Monitoring and managing the Hadoop cluster.
• Working on setting up Hadoop multi node clusters, pig, Hive and HBase.
• Using Sqoop extensively to import data from RDMS sources into HDFS.
• Performed transformations, cleaning and filtering on imported data using Hive and loaded final data into HDFS.
• Involving in creating external table, partitioning, bucketing of table.
• Responsible for creation and setting up of environment and re-configuration activities.
• Created reports for the BI team using Sqoop to export data into HDFS and Hive.

I hereby declare that all the information furnished above is true to the best of my knowledge.

[nom]