Big Data Engineer
What is a Big Data Engineer?
A Big Data Engineer is a professional who is responsible for designing, building, maintaining, and testing the infrastructure required for processing and storing large-scale data sets. The main role of a Big Data Engineer is to develop, construct, and maintain scalable and efficient data processing systems, ensuring that they meet the specific needs of an organization.
What is the responsibility of Big Data Engineer?
As a key member of the technical team for Product development, you will be expected to define and contribute at a high-level to many aspects of our collaborative agile development process:
- Expert software implementation and automated testing
- Promoting development standards, code reviews, mentoring, knowledge sharing
- Improving our Agile methodology maturity
- Product and feature design, scrum story writing
- Build, release, and deployment automation
- Product support & troubleshooting
what knowledge should have as a Big Data Engineer?
- Crystal clear demonstrated experience in Java Programming skill.
- Should have a deep understanding of Enterprise/Distributed Architecture patterns and should be able to demonstrate the relevant usage of the same
- Turn high-level project requirements into application-level architecture and collaborate with the team members to implement the solution
- Strong experience and knowledge in Spring boot framework and microservice architecture
- Experience in working with Apache Spark
- Solid demonstrated object-oriented software development experience with Java, SQL, Maven, relational/NoSQL databases and testing frameworks
- Strong working experience with developing RESTful services
- Should have experience working on Application frameworks such as Spring, Spring Boot, AOP
- Exposure to tools – Jira, Bamboo, Git, Confluence would be an added advantage
- Excellent grasp of the current technology landscape, trends and emerging technologies
The additional advantage of Big Data Engineer.
- Ability of taking up tasks independently and willing to be individual contributor when required.
- Working experience or knowledge of Scrum/Kanban and SAFe agile processes
- Should have an excellent communication skill (Verbal & Oral)
- Experience with Elastic Search would be plus
Big Data Engineers work with huge volumes of structured, semi-structured, and unstructured data. All the above features or aspects should hold to get succeed in the field of Big data engineer.
Overall, a Big Data Engineer is a crucial role in any organization that works with large-scale data sets, ensuring that data is processed, stored, and accessed efficiently and securely.