Hadoop Administrator

Job ID:
Chicago, IL
Information Technology, Telecommunications, Array
$125,000.00 per year
Zip Code:
Employment Type:
Full time

Job Description:

As a Hadoop Administrator, you will be responsible for the design, implementation, Security and on-going support of Company's Hortonworks Hadoop Distribution on AWS. You will partner with Data Scientists as well as product stakeholders to create an advanced analytics framework.

Key Responsibilities

  • Manage large scale multi-tenant Hadoop cluster environments residing on AWS.
  • Handle all Hadoop environment builds, including design, security, capacity planning, cluster setup, performance tuning and ongoing monitoring.
  • Perform high-level, day-to-day operational maintenance, support, and upgrades for the Hadoop Cluster.
  • Research and recommend innovative, and where possible, automated approaches for system administration tasks.
  • Creation of key performance metrics, measuring the utilization, performance and overall health of the cluster.
  • Deploy new/upgraded hardware and software releases and establish proper communication channels.
  • Work with appropriate stakeholders to ensure we have solid capacity planning and can manage our TCO.
  • Ability to collaborate with product managers, lead engineers and data scientists on all facets of the Hadoop Eco-System.
  • Ensure existing data/information assets are secure and adhering to a best in class security model.

Job Qualifications

Technical Experiences Required

  • Possess at least 3 years of managing a multi-tenant production Hadoop environment;
  • A deep understanding of Hadoop internals, design principals, cluster connectivity, security and the factors that affect distributed system performance;
  • Proven experience with identifying and resolving hardware and software related issues;
  • Experience with Hortonworks DataPlatform;
  • Knowledge of best practices related to security, performance, and disaster recovery;
  • Expert experience with at least two of the following languages; SQL, Python, Java, Scala, Spark or Bash.

Additional Experiences Preferred

  • Experience managing Cloud Services (IaaS, PaaS). AWS Certification is preferred;
  • Experience with a number of different Real Time integration tools such as Apache Flink, Kafka, Storm or Flume. Leveraging Hortonworks DataFlow is preferred;
  • Experience with Git, Jenkins, Ansible a plus
  • Experience with complex networking infrastructure including Firewalls, VLANs, and load balancers.
  • Experience as a DBA or Linux Admin

Personal Experiences Required

  • Proven ability to lead multiple high priorities initiative with aggressive timelines leveraging an agile framework.
  • Comfortable performing in a fast paced, dynamic and ambiguous business environment.
  • Ability to concentrate on a wide range of loosely defined complex situations, which require creativity and originality, where guidance and counsel may be unavailable.
  • Demonstrated a product mindset with an ability to set forward thinking and direction.
  • Excellent listening and communication skills.

o Must be able to communicate technical information clearly, able to translate' between diverse groups of technical and non-technical individuals.

o Ability to synthesize large amounts of complex data into meaningful conclusions and present recommendations to a vast array of individuals.

o Partner with business units in order to improve the effectiveness of business decisions.

o Ability to structure documents to effectively communicate with senior leadership and drive alignment and decision making.

  • Strong problem solving skills with the ability to offer why one technical solutions is more preferable than another.
  • Ability to analyze and interpret complex concepts.
  • Ability to lead a group of employees (directly or indirectly).
  • Ability to work in a team environment in a collaborative manner.
  • Proven track record of sound, effective decision making.
  • Bachelor's Degree in Computer Science/Business/Data Sciences required, Master's Degree Preferred.
Company Info
Request Technology - Anthony Honquest