• Devised a solution that enriches 6TB of raw logs per day with data across distinct projects using Amazon trace IDs. The resulting table is aggregated and visualized in a holistic AWS Quicksight dashboard. To achieve this, I built upon an existing architecture and simplified it from 12 inter-connected components (SQS queues, Lambdas) down to 5. I then automated the pipeline to the point of live data ingestion (1 hour delay) • Profiled and optimized the main analysis Lambda (avg. runtime improved from 15 to 8 secs), which, at the scale of 720.000 files processed per day, led to savings in Amazon Lambda worth approx. $30.800 annually • Used supervised (decision trees) and unsupervised (PCA) machine learning techniques to unlock business insights from the newly-acquired data which fundamentally improved the team’s understanding of some of its operations

• Used Apache Spark, Airflow, Prometheus, Grafana and Tableau to create a big data (∼100Gb compressed) processing and analysis solution from scratch featuring daily scheduling of jobs and real-time visual monitoring • Fixed skewed executor workloads (processing time dropped from 52mins to 12mins), replaced manual aggregation over periods longer than a day with reaggregated HyperLogLogs, used a working set for merges up until a threshold to take advantage of memory-optimized instances, compacted sketches before storage (from 5.1Gb to 809.2Mb) - all of which led to savings in Amazon S3 & EMR worth approximately $10.000 annually

Outline • The RESTful Web App that I worked on (C#, .NET Core) functioned as a robust introduction to front-end development: behaviors (AngularJS), styling (CSS, Bootstrap), document structuring (HTML), and the Open API specification (Swagger). I also furthered my knowledge of backend MVC engineering (designed 2 controllers and 1 model class, both spanning over 20 methods) • Improved the final runtime of my project from an initial 3 mins 47 secs to a final 6 secs, after careful reconsideration of design choices: switched from a depth-first search to a breadth-first search (twice as fast in that context), grouped as many database queries together etc. • Operated in an Agile Development team with a Scrum master, retros, and standups • Leveraged the Secure Code Warrior platform to build knowledge around common software vulnerabilities: XML injection, SQL injection, Cross-site scripting (XSS), insecure direct object reference, OS command injection, remote file inclusion, and how to protect against them

Worked on Kalignite Terminal Handler (KTH), a project which improved my object-oriented programming skills (C#) and allowed me to build knowledge around file administration (through a Console Application), user interface design (Windows Presentation Foundation), and automated report documents processing and generation (Excel Interop)