Datameer Blog post
5 Big Data Job Descriptions to Hire an All-Star Team
by Erin Hitchcock on Feb 27, 2018
So you’ve made the business case for hiring a big data team to spearhead your company’s analytics initiatives – now what? Once you’ve identified the operational value proposition and business insights that big data offers you, then (and only then) is it time to add the talent that can deliver on those expectations.
Download Your Free Ebook on the Big Data Journey
Finding people with the correct skillset to address your organization’s needs can be a daunting task and there are a few key individuals you’ll need to help guide your big data journey to success. However, finding and convincing these people they’re the right fit for the role can be a challenge if you don’t know how to ask for what you need.
In an effort to speed up the process of attracting the perfect match for your growing team of big data all-stars, we’ve assembled seven detailed career descriptions to assist with the recruitment process for the essential roles you’ll need:
1. Data Engineer Job Description
The data engineer is a critical member of your big data team because they are dedicated to the fundamental process of capturing, storing and processing your data. A great data engineer will not only understand and organize your data, but will also curate it and find ways to get it to the right people (your business analysts). They’ll develop the overall architecture that helps analyze and process data in the way the organization needs it, and they’ll make sure those systems are optimized effectively.
We are looking for a skilled Data Engineer to join our analytics team. The ideal candidate has an eye for building and optimizing data systems and will work closely with our systems architects, data scientists, and analysts to help direct the flow of data within the pipeline and ensure consistency of data delivery and utilization across multiple projects.
- Work closely with other data and analytics team members to optimize the company’s data systems and pipeline architecture
- Design and build the infrastructure for data extraction, preparation, and loading of data from a variety of sources using technology such as SQL and AWS
- Build data and analytics tools that will offer deeper insight into the pipeline, allowing for critical discoveries surrounding key performance indicators and customer activity
- Always angle for greater efficiency across all of our company data systems.
- Graduate degree in Computer Science, Information Systems or equivalent quantitative field and 5+ years of experience in a similar Data Engineer role.
- Experience working with and extracting value from large, disconnected and/or unstructured datasets
- Demonstrated ability to build processes that support data transformation, data structures, metadata, dependency and workload management
- Strong interpersonal skills and ability to project manage and work with cross-functional teams
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Experience with the following tools and technologies:
- Hadoop, Spark, Kafka,
- Relational SQL and NoSQL databases
- Data pipeline/workflow management tools such as Azkaban and Airflow
- AWS cloud services such as EC2, EMR, RDS and Redshift
- Stream-processing systems such as Storm and Spark-Streaming
- Object-oriented/object function scripting languages such as Python, Java, C++, etc.
2. Systems Architect Job Description
This team member is responsible for how your big data systems are architected and interconnected. Their primary value to your team lies in their ability to leverage their software engineering background and experience with large scale distributed processing systems to manage your technology choices and implementation processes. You’ll want this person to build a data architecture that aligns with the business, along with high-level planning for the development. He or she will take into account various constraints, adherence to standards and differing needs across the business.
We’re seeking a Systems Architect with strategic experience who will research, model and integrate architectures to align with business strategy, goals and objectives. The Systems Architect will create architectural principles to support business goals, and develop IT frameworks that support business applications.
He or she will understand and model key business capabilities, processes, relationships and impacts of technology on business goals. The Systems Architect will also formulate conceptual architectures and communicate architectural vision, goals and design objectives to multiple audiences.
- Determine database structural requirements by analyzing client operations, applications and programming; review objectives with clients and evaluate current systems
- Develop database solutions by designing proposed system; define database physical structure and functional capabilities, security, back-up and recovery specifications
- Install database systems by developing flowcharts; apply optimum access techniques, coordinate installation actions and document actions
- Maintain database performance by identifying and resolving production and application development problems, calculating optimum values for parameters; evaluating, integrating and installing new releases, completing maintenance and answering user questions
- Provide database support by coding utilities, responding to user questions and resolving problems
Extensive experience in:
- Agile and/or Scaled Agile Framework (SAFE) Software Delivery Life Cycle
- API Creation and Management (REST, SOAP, etc.)
- Service-oriented architecture
- IT standards, procedures, policy
- Application delivery process
- Business process modeling
- IT environment
- Product and vendor evaluation
- Information security management
Working experience in:
- Information management
- Information security architecture
- The Open Group Architecture Framework (TOGAF)
3. IT/Operations Manager Job Description
The IT/Operations Manager is a valuable addition to your team and will primarily be responsible for deploying, managing, and monitoring your big data systems. You’ll rely on this team member to plan and implement new equipment and services. He or she will work with business associates to understand the best technology investments to address their processes and concerns—translating business requirements to technology plans. They’ll also work with project managers to implement new technology and be responsible for a successful transition to operations.
The IT/Operations Manager is an integral part of the big data team. He or she will ensure that production IT systems and services continually meet business needs in a reliable and efficient manner. This role requires strong leadership and technical skills around incident and problem management, automation and monitoring, system integration and IT operations. The IT/Operations Manager will assess and analyze complex data problems and build solutions that meet objectives while working closely with key clients and team members.
- Manage and be proactive in reporting, resolving and escalating issues where required
- Lead and co-ordinate problem management activities, in addition to continuous process improvement initiatives
- Proactively manage our IT infrastructure
- Supervise and manage IT staffing, including recruitment, supervision, scheduling, development and evaluation
- Verify existing business tools and processes remain optimally functional and value added
- Benchmark, analyze, report on and make recommendations for the improvement and growth of the IT infrastructure and IT systems
- Develop and maintain a corporate SLA structure
- 8+ years of experience in Operations or Support leadership role (e.g. Team Lead, Manager, etc.)
- Strong working knowledge of physical IT infrastuctures (e.g. Servers, SANs, Networking, etc.)
- Basic knowledge of CR infrastructures (e.g. UPS, Generator, AHU, etc.)
- Exposure to monitoring products such as Solarwinds, NewRelic, Splunk and Omegamon
- Proficiency in systems administration, telecommunications, monitoring and backups
- Strong project management skills
- Strong interpersonal skills and a customer service-oriented mindset
4. Business Analyst Job Description
This role is essential to success for your big data journey. The Business analyst is the link between a company’s data program and its business objectives. Business analysts help businesses extract value from their big data systems in a cost-effective way by determining the requirements of a project or program, and communicating them to stakeholders and partners. Business analysts are particularly important to your venture due to their intimate industry and company knowledge. This allows them to analyze business-level data with a critical eye for perceiving actionable insights that will add value to your organization.
The Business Analyst will work with stakeholders from all business units and related third parties to define and document business processes and software requirements for various initiatives. The Business Analyst evaluates and validates new and existing functions, identifies and analyzes end-user issues and provides support through standardized processes, tools, workflow analysis and testing to ensure end-user requirements are fulfilled.
- Identify business trends utilizing real data, compile analysis reports that are delivered to developers and then follow-up on all results
- Interact with the business to identify, capture and analyze business requirements
- Identify areas of process improvement
- Develop functional specifications in a team environment, as well as derive use cases where appropriate
- Perform data analysis including data mapping, report analysis, interface definitions
- Develop a test plan including functional QA, integration testing, string testing, ICAT, ECAT, etc.
- 5+ years of associated work experience
- Advanced SQL database management and maintenance skills
- Detailed analytical abilities
- Written and verbal communication, preferably with technical writing skills
- Strong experience in user testing and project management
5. Data Scientist Job Description
The data scientist will be a consumer of your system. These individuals help create machine learning models that your team will then deploy into the system to get to the business operations. The data scientist’s understanding of statistical analysis principles will allow them to pull information from your data that will help answer important business questions and lead to valuable insights. Keep in mind, however: having these users up and running should not come before having infrastructure to support your company’s data.
Our leading data management and integration organization is currently in need of a Data Scientist to join our fast-growing team. The ideal candidate will be intricately involved in running analytical experiments in a methodical manner, and will regularly evaluate alternate models via theoretical approaches. This is the perfect opportunity for the successful candidate to become a part of an innovative and energetic team that develops analysis tools, which will influence both our products and clients.
- Initiate and participate in projects in the area of prediction, optimization, and processes using advanced statistical / mathematical approaches, in the enterprise environment
- Implement most recent algorithms and approaches for machine learning in collaboration with our researchers
- Design best architecture and select the most appropriate modeling techniques and data visualization for big data analysis
- Iteratively test, refine and improve the models
- Collaborate with product management and engineering departments to understand company needs and devise possible solutions
- Optimize joint development efforts through appropriate database use and project design
- Learn quickly new mathematical or technical methods
- Ph.D. or MS in Computer Science, Electrical Engineering, Statistics, Applied Math or related quantitative topics
- Able to understand various data structures and common methods in data transformation
- Experience in statistical modeling, machine learning, or data mining practice
- Experience in SQL, relational databases, database concepts, dimensional modeling and database design
- Strong analytical and quantitative problem solving ability
- Excellent communication, relationship skills and a strong team player
Building your all-star big data and analytics team won’t be the first or last step in your big data journey. If you’ve gone about the process correctly, then your journey began with understanding the business initiatives that drive the need for a comprehensive approach to big data strategy in today’s increasingly data-centric marketplace.
But since data-driven technology is constantly evolving, don’t forget that your strategy for extracting value from the big data machine will also need to adapt to keep pace. It’s perfectly acceptable to grow your team to accommodate changes in the big data and analytics landscape, but be mindful that you aren’t simply increasing body count without a clear plan of how this will add to your bottom line. A good rule of thumb is to consider team expansion once you begin to see team members routinely performing tasks outside of their initial function that need to become dedicated specialist roles.
And, while it’s tempting to embark on a never-ending search for the unicorn candidate whose experience reads like a laundry list of all the most bleeding-edge big data technology – understand the importance of hiring to fill gaps for the capabilities that you need as opposed to attempting to fulfill an overly ambitious wish list. As with most hiring initiatives, the end goal should have efficiency in mind. Don’t be afraid to invest in the people that will meet your business needs and provide the greatest returns, without overextending their usefulness.