Project Kampong is an online collaborative platform that aims to map assets across communities and create a documentation platform for the Social Sector to consolidate needs, resources and gaps. I led a team of 3 developers to build the project's backend system, where we worked on implementing data models, RESTful APIs from product requirements. We also explored the use of GraphQL and WebSockets (socket.io) while iterating on the product. We also worked with the National Volunteer and Philanthropy Centre (NVPC) to better understand the volunteering landscape and user requirements.

In a team of 5, we built a traffic congestion prediction model using Python and PySpark. Using the Bing API, we collected traffic data in Singapore. We then combined this with traffic image and weather data from the Singapore government agencies. With these data points combined, we then used the data to build a model that predicts the likelihood of traffic congestion for the day. To handle the large input data of around 7.5GB/day, we used AWS S3 as our data lake, and implemented an ETL pipeline in PySpark hosted on an EMR cluster on AWS. We also used Redshift to store the data in a database and to query and train the model. My role in this project centered around the ETL pipeline, where I designed the deployment architecture and implemented ETL scripts for our various data sources. We managed a model accuracy of 88% using a random forest classifier.

I built a Telegram bot to track the availability of COVID-19 vaccinations in Singapore. I implemented a pub-sub mechanism to notify users of the availability of vaccinations when they are available. Using Redis, I stored subscribed users and I also implemented a caching mechanism to reduce the number of API calls to the source API exposed by the government.

To help a company retrieve it's app DAU from its CMS daily, I automated this process by building a Selenium based web scraper to retrieve the relevant metrics and send an email to the relevant stakeholders. I took the opportunity to explore AWS serverless deployment methods including AWS Lambda and Fargate. Eventually, I implemented a solution using Fargate and EventBridge to run the scraper daily. I had chosen Fargate as we had a containerized implementation that was already running locally. There was a lower risk of a difference in behaviour as the same container image would be used in deployment.