Summer Updates

Time: 05/11-07/26, 180+ hours

Topics: N/A

Tools: Elasticsearch, Digital Ocean, Node.JS, Laravel.

Summary:

I spent the summer of 2020 with my mothers’ side of the family in Toronto. I moved from my aunt's home in Markham to my grandparent's home once school ended in May. I applied for an internship scholarship from Wheaton’s Center for Vocation and Career and because of that funding I was able to continue working for Wunderite throughout the summer. The blanket of changes to a regular life that COVID-19 swaddled my life in became so regular. Day to day happenings at my grandparents eventually became routine and that routine became comfortable for me. 

My grandparents lived in the same house as my mothers’ second sister's family. My grandparents, aunt, uncle, 2 cousins, and I got along very well for the 4 months I stayed there. Looking back, It was such a rare and precious moment to live with them and learn from them. 

I was grateful to be in the same timezone as Wunderite in Boston and was able to attend morning scrum meetings, weekly 1 on 1 call with Peter and Joe, sprint planning meetings, and tech calls with Rick on Wednesdays without too much interruption. Amidst the GitHub pushing and pulling, zoom calls, and obligatory frustrations with learning new tech stacks, I did find time to enjoy the outdoors on my aunts’ Schwinn, learn to drive in my grandfather’s Subaru, spend time with my cousins and fall asleep with books on my face in my hammock. 

My summer with Wunderite was different from my Spring internship for 2 main reasons. One, TechStars was over so the team could refocus and organize new routines for agile development. Two, I wasn’t as green as I had been in January and was a lot more confident taking on tasks each week and discussing solutions to problems with the rest of the development team. 

During those 4 months I accomplished the following with Wunderite:

1. I added an About page on the marketing site that provided info on Joe and Peters’ background as founders of the company
2. With some help from Cyril, I added a green status icon to the footer of their marketing site that fetched and was linked to data from the OhDear API that shows the current status of the Wunderite web app. 
3. I reformatted their Terms & Privacy page to pull content from Prismic so that it could be updated more easily and was styled more in line with the rest of the site.
4. I added functionality to intentionally leave a field blank on the questionnaire survey in Laravel using VueJS and the PDFTK API on the web app.
5. I created a spreadsheet to score my scraping confidence of all 50 US States based on criteria such as Table fields, minimum search query length, and result length.
6. I wrote scraping tools for the government corporation divisions of Massachusetts, New Jersey, and Rhode Island in NodeJS using Google Puppeteer and MySQL. These tools scrapped over 3.6 million business entities with crucial information such as addresses and status as well as the URLs to more specific data per business using headless browser technology and fail-safe restart-capable async functions. My final iteration of the MA business entity scraper was capable of crawling the .gov corporation site with more than 2 million business entities in less than 24 hours, copying and parsing the rows into a private SQL database all from my personal machine.
7. I set up an ElasticSearch index on a Digital Ocean Ubuntu server to host the data I scraped and integrated the index with Wunderite’s web app to create a working business entity search engine. This tool would be helpful for insurance agents to pull the useful information from businesses without going through the tedious process of searching state corporation division sites. I created a document to track my learnings and help future developers do the same.

Scraping business entities from government websites was a real party. I’m grateful for the opportunity to learn the Elastic stack and implement it on a Digital Ocean server.

My summer in Toronto with Wunderite taught me to be patient, that time spent in front of a monitor must be balanced with time spent outside, that working for a startup is exciting and enables you to learn a lot—fast. 

I want to thank Peter and Joe for being such excellent mentors and supervisors, pointing me in the right direction and helping me however they could; Phil for being such an effective and determined sprint leader; Cyril, Adam, and Chris for always answering my questions and reviewing my work.