Teamwork, Emotions, Children, and Software Engineering all in one Place

The Organization and Product:

S
tory Squad is an application, web-based, and mobile, designed with children ages 8–12 in mind. It was started by a 6th-grade teacher, Graig Peterson, and a Data Scientist, Darwin Johnson in July 2019. They started Story Squad because they want to bring creativity back to the forefront again and combat screentime fatigue in students. This application is a collaborative world-building game in which children can tap into their creativity to illustrate and author a side story based on part of a story they just read. This application is going to be a great tool for the education sector when it is rolled out to the public.

A quick introduction from the founder of Story Squad

My First Impression:

W
hen I first got access to the roadmap for this project, it was really overwhelming. There was so much information to comb through from previous developers. I was given access to information to help me learn about Story Squad, what work has already been done, and my team’s objectives. Things started to become clearer when I was able to attend the first meeting with the stakeholders for this project. Being a mother of two boys who are 11 and 13 years old, I was super excited to be working on such a meaningful project, and that excitement never did fade. I really enjoyed the part where they let us play the game ourselves, just like we were a child again.

The top drawing is my 11-year-old’s and the bottom is mine.

This was a really fun experience for me because not only did it let me see the application from the user’s perspective, it also gave me some quality time with my boys. We read the story together, taking turns reading it out loud, and then we all did an illustration of a different scene from the story. They did not have enough time to write the side story but they really enjoyed reading the story and drawing the illustration with me. Since I have started my journey to become a Data Scientist, this was the first time they were able to work with me on something related to my journey. That is a moment I will treasure, thanks to Story Squad.

Our Main Task:

A
lot of work was put into this codebase before my team came onto this project. However, no one has implemented any testing yet on the inputs or outputs of the active endpoints. There were two “test” files in the code base, but there was nothing really implement yet. Whoever started those “test” files must have done so with the intentions of implementing them but probably ran out of time before they were able to. So our team was tasked with creating a testing suite. One thing that concerned us all was that none of us have had any experience with writing tests before this experience.

Challenges:

Challenge 1:
What testing method to use?

There were three options that our team was looking at for the method to use for testing.

1. TestClient which is built-in to FastAPI (this was the framework used to build out the data science API for this project)
2. unittest — a built-in library in Python — widely used in the industry
3. pytest — a separate library that would need to be installed, but it is versatile and can be used in combination with unittest

There was a little confusion on which testing method would be better for this project. Our team was really leaning more towards using Testclient since the API was built using FastAPI. To clear up this confusion, we asked the stakeholder which method they would prefer we use. The stakeholder responded with, “unittest, unless you can convince me otherwise.”

After getting that response from the stakeholder, we talked about it as a team. As we discussed it, we came to the conclusion that unittest would be the better option for a few reasons:

• More flexibility — with unittest if for any reason, future developers decide to change the framework used for the API, it would be easier to convert the testing for the new framework.
• Used industry-wide — unittest is used more frequently in the industry for testing code vs TestClient, which means it will be easier for someone new to pick up where we left off.
• The Preferred Method — being this was the preferred method of the stakeholder, we assumed this is because they are already familiar with unittest.

Challenge 2:
Setting up our environment to run the API locally on a windows machine

All of the members of my team had Windows 10 machines. The code was originally built on a Linux or Mac machine, so there were several issues with setting up our environments. There were already requirements.txt, Pipfile, and Pipfile.lock files in the codebase when we received it.

• The first problem we faced was setting up our pipenv environment. We kept having dependency issues that were preventing the Pipfile.lock from locking properly on our machines.

✓ Solution:
After doing some research, I found the solution to our problem. Some of the dependencies that were used in this project are not compatible with Python 3.9. I have several versions of Python on my machine, but since 3.9 is the latest version, the pipenv defaulted to using it. To make it work properly, I had to specify the Python version I wanted to use, like so: pipenv --python 3.8 install --dev .

The next problem we faced was a path issue. This was the main issue for those of us using a Windows 10 machine. We were getting a FileNotFoundError message for the JSON object that should have been created when we accessed the Google vision API when deploying our API locally.

✓ Solution:
This blocker was a little harder to figure out. It took a lot of research and trial and error to find out what path worked for my machine without having to do the absolute path. There were a few steps that I needed to take to solve this issue:

1. Needed to create a new directory: tmp
2. Then create an empty file inside that directory: google.json
3. Finally, needed to change all paths that called on this JSON object from /tmp/google.json to ././app/tmp/google.json and another team member had to make his path app/tmp/google.json .
4. We just had to make sure not to push any of the changes we made for this to GitHub because the other path still works on a Windows machine if you use Ubuntu WSL or a Linux or Mac machine. We were told that once it is deployed, it will be deployed on AWS Elastic Beanstalk using a Linux system, so it needed to remain as is.

Challenge 3:
How to use unittest to test inputs & outputs of my endpoint viz/inegraph?

First, I needed to do my research on how to implement the unittest methods. Through my research, I found a lot of resources but the one that helped me get the best understanding was the following video on YouTube:

This tutorial helped me the most.

I also took a course on LinkedIn Learning, Unit Testing and Test Driven Development in Python, which helped to give me a more in-depth understanding using TDD (Test Driven Development). TDD does not necessarily apply in this situation, but it did help with my understanding of how to implement the test.

Before I could dive into the code though, I really felt like I needed a clear understanding of exactly what my endpoint was doing. Now was the time to search through the large codebase and trace all the files my endpoint touched for it to function properly. I decided to create a “map” or flowchart on whimsical to visualize my endpoint connections.

My endpoint connections — View on Whimsical

The first thing that I implemented was the logic for setting up the TestCase class. Within the class, I add a class method for my setup class to initialize my dummy data in the pydantic model format for my endpoint. This code will run once at the beginning when the TestCase class is called. Then I wrote the logic for the setup method that would break down the dummy data before running each test within the TestCase class.

Example of the logic for my setup class and setup methods

The next step was to test for the edge cases of empty or improper data input.

Tests I implemented to test for empty data and invalid data being inputted.

A fellow teammate came up with a function that we both used to return a boolean to verify if the output was a JSON object. Then I created the test for verifying the proper boolean response using that function.

The function that a fellow teammate, Thomas Shorts, provided and the test I created to verify the proper boolean was returned

To be able to test that the JSON object had the proper data outputted in it was a little bit tedious. The JSON that is outputted for this endpoint is for a Plotly line graph. So it was a fairly large file to look through to find the locations where the input data was stored. I then tested the output to ensure that the data from the input files were in the proper place within the output JSON.

The final test that I implemented tested the response status code. This one was a little more difficult to implement. My first instinct was to use the mock library from the unittestmodule and I did attempt to do this in several ways. We were running out of time and this was the last test needed to be able to have a fully functional TestCase class.

The only way that I could see how to implement this test was to call on the API directly, which also means the user will need to have the API running locally in the background to run the test. This is not ideal and using the mock library will be the better way to go when there is more time for research or someone with more knowledge of mock is working on it.

I implemented the test for testing the status code using the data outputted on the Swagger UI for the “Curl” command. Then I used the request library to send a post request to the endpoint with properly formatted and improperly formated dummy data. All I needed to do next was to get the status code and assert that I was getting the proper status code for the dummy data. I shared the solution with my team because we were all having issues with this blocker.

To implement some of the testings on the inputs, I did need to also add some edge cases to the function which created the line graph and outputted the JSON object. I added edge cases for an empty ScoreHistory list, an empty string passed in for StudentName, and default data.

Working on a Team vs Independently:

This was not the first time that I worked with a team on a project but this was the longest time period, two months. There are advantages to working on a team vs independently, but there are also some disadvantages.

Image courtesy of Dreamstime.com

Some ways working on a team are useful:

• Ability to delegate tasks to deliver a completed project/feature quicker
• Diversity of knowledge helps to bring new ideas to the table
• Sharing of multiple resources
• You have multiple “sounding boards” to bounce your ideas off of
• If you are on a good team, your teammates are there to support you in any way you need support

Image courtesy of depositphotos.com

Some ways working on a team can be hinder some:

• With the diversity of knowledge, this could cause disagreements amongst the team, which could actually be good and bad, depending on the situation and how everyone handles it.
• Different personalities can collide and cause conflicts within the team. This could cause an uncomfortable working environment for others.

While working on this project, I feel I really lucked out with the team I got put on. For the most part, our team got along really well. Just like with any group though, there were moments of tension and disagreements. Through this experience I have learned that what makes a team effective and productive is not being the perfect team, but how you handle the imperfections and move on from them.

One example of a disagreement we had as a team and how I chose to handle it:

No one on the team has done any kind of API testing prior to this project, we were all “green” and needed to learn. We had set out to do our own research but one of our teammates was on the ball and found a great resource on testing, which was a series of short videos on YouTube called API Testing using Python. He shared the link to the playlist with the team just a short time before our meeting, so no one had a chance to watch it before we met.

While we were in our meeting, the teammate who posted the series on testing mentioned that he felt we would only need one file for all the tests we needed to complete for the five endpoints we will be testing. He said that after watching the series he posted, it looked like it was going to be fairly easy to implement testing and it should only take a few hours of pair programming to knock out all the tests. I disagreed with this statement.

I was still “green” at testing myself, but from the little bit that I remembered from the very brief introduction we got to testing, this testing suite was going to be more than you would want in one file. Since I was not very confindent that this was correct at the time, I did state my opinion but also agreed to watch the video series so I could see what he was talking about. Maybe he found something that would make that posible.

I watched the video series the next day. While it was a good testing tutorial, it was not feasible for the project we are working on because it never touched on unittest, it only went over using pytest on a less complicated code base than what we have.

So I set out to do my own research and remembered that someone sent me a video on unittest a while back. I found that video, which was the one I mentioned earlier (Python Tutorial: Unit Testing Your Code with the unittest Module), and watched it and took notes. I found it to be very useful for the problem at hand.

So I shared the video that I found helpful and all the notes I had been taking on both of the videos with my teammates. I asked them to all watch the video and take a look at my notes. I did not want to step on anyones toes here, but I wanted to also make sure that we were all on the same page and that the product we produced was what the stakeholders were looking for.

Here is the link to my notes I shared with my team.

What I Learned:

Over the past two months of working on this project, I have learned a lot! The biggest thing that I believe I learned, which will carry over into my career, is:

You can be on the best team in the world and the majority of the time you all get along and can quickly come to an agreement on most things. No matter how hard everyone tries though, there will always be bad days, just like with any relationship. The key to a productive and effective team, is how you handle those adversities and move on from them.

You also need to keep in mind that everyone will always have something going on in their personal lives that you may know nothing about, so don’t jump on them for the slightest mistake because you have no idea what struggles they are dealing with personally.

I definitely learned a lot about testing your code and how important it is to have testing in place. Working with the FastAPI framework and pydantic models, there are a lot of “validators” you can add to your code that act like testing and they will catch a lot of the edge cases for you. However, they are not a replacement for testing.

The main reason you want to implement a test is to test the functionality of all the functions within your codebase to ensure they are working properly. You need to make sure that if someone bypasses the API and enters the data directly that you have code in place to catch those edge cases too.

Resources:

• Link to line_graph.py
• Link to test_visualization.py
• Learn More about Story Squad
• Graig Peterson on LinkedIn
• Story Squad on LinkedIn
• Python’s built-in unittest documentation
• FastAPI’s TestClient documentation
• pytest documentation
• Python Tutorial: Unit Testing Your Code with the unittest Module
• LinkedIn Learning: Unit Testing and Test Driven Development in Python
• Convert Curl command to Python code

Connect with Me:

• LinkedIn: Joanne Middour
• GitHub: jmmiddour
• Portfolio: joannemiddour.com