hackgt / api

Backend API powering HexLabs' services

api

HexLabs' backend API with multiple services and a custom API gateway. Please visit our Wiki to view all of our documentation.

api's Issues

Update api call for pass through error

If an apiCall throws an error, the error should be passed through better.

Switch prize-item to swag-item

Just more clear

Update expo permissions

Change expo permissions to use api repo permissions instead of the user permissions from the old expo library. This will help keep everything managed together

Update the docs site with new endpoints

I have assigned each of you to one of these services. You should be updating the docs site by adding the routes that aren't already there to the swagger-output.json file in the /docs folder, You may find this editor useful or you can just run it locally.

• users - @nimitadesh [COMPLETED]
• files and notifications- @rahikot
• hexathons - @AaditT
• registration - @Anupama1912
• auth and hardware - @thaneesh-babu [COMPLETED]

Create a PR with the changes referencing this issue.

Add interaction summary routes

The interactions service is responsible for logging all interactions that occur at our events (for ex like checking in, completing a scavenger hunt puzzle, etc). As such, update the / GET endpoint to add a new filter query to filter by identifier. This identifier is a string used to identify an interaction.

Additionally, add a new route /statistics GET that has a mandatory request query called hackathon. This route should summarize the interactions data from the database, and return the matching interactions that are a part of this hackathon. It should be in this format of a json {}.

It should be:

{
  identifier: {
    type (type directly from the database)
    count (# of interactions logged with this identifier and hackathon)
    firstTimestamp (timestamp of first interaction under this identifier and hackathon)
    lastTimestamp (timestamp of last interaction under this identifier and hackathon)
  }
}

You can use the interactions.api.hexlabs.org prod route to look at sample data

Create team management routes

In the users service create a new model called Team. This data structure and routes will be used to manage teams for an event. Each team is specific to a certain event, and users can join teams. Each user can be part of multiple teams, but only one team per event they are attending.

The team data structure should have these fields

• name (string, required)
• event (Object id linked to an event, required)
• members (array of strings linked to a userid, required, with default no members)
• description (String, optional)
• public (boolean, required, default: false, determine if a team can be publically found and joined through team formation)

Create the following routes:

create team (post "/teams")

• pass in the above data to create a new team for an event
• the first member should be the req.user
• when creating a team, the name needs to be unique for that event

get teams (get "/teams")

• get all the teams
• should additionally have a search param option to filter by event

join team (post "/teams/join")

• req.body should contain event and name, and then you should be able to get the team from this info
• first ensure the team exists
• then adds the current req.user to the team if they aren't already a part of it
• should handle the case when a user is already part of another team for that event as a user can only join one team per event
• should handle case when user is already part of the team, and only add them once

leave team (post "/teams/leave")

• req.body should contain event and name, and then you should be able to get the team from this info
• first ensure the team exists
• then checks that the user is part of the team, and removes them from the members field if they are

Setup better service to service calls

Setup a better way to do service-to-service calls without permission restrictions. Like for example, sponsors need access to resumes, but it's annoying to do cross communication permissions.

Add registration branch model

For the registration service (in /services/registration), this service will handle routes and services related to registration. This service will handle creating new branches, applications, etc.

For this task, create the mongoose model for a branch.

For the branch model, here are the fields we want to include:

• name (type: string, required: true)
• type (type: string, required)
• settings
  • open (type: date, required)
  • close (type: date, required)

Make sure to create both the typescript schema and the mongoose schema

Create interaction models and routes

For the interactions service, this service will handle the majority of the logging and data collection for when a participant attends an event (like workshop, mini-event, etc) and will be called an "interaction". Currently, a lot of the backend old interactions routes lives in the yac repo, and also has a lot of code handler for virtual events.

Reference this function and see how the current implementation is. When making the post request to create an interaction, there needs to be a lot error handling to make sure the event has started. Additionally, if an interaction is already created for a particular user and event, a new interaction shouldn't be created. There is other info in there that we have removed for this service.

For this issue, create a new model called EventInteraction that stores the event uuid from cms, the userid as a string, and timeIn to the event.

For the routes, create a get route to get all interactions, post route to create a new interaction, and put route to edit an interaction.

Create cloud functions to integrate with GCloud Identity Platform

Our new auth provider Google Cloud Identity Provider power login.hexlabs.org. They provide the ability to add HTTP Cloud Functions to run when a new account is created and before each sign in. We would like to use these functions to do things like sending a welcome email and setting permissions.

Here is a screenshot of what the page looks like.

For this issue, explore Google Cloud Functions, and create a folder inside our api repo to hold them. Try and setup a package.json in this folder, and integrate it with Yarn Workspaces and tsconfig.json to integrate with the common folder. Ensure that you can get the basic setup working.

Move over points backend and integrate into api

Looking at the points bakckend here, move over all the relevant files to the interactions service. This should include all the relevant schema models in its own file, all the server routes. Additionally, since the old version of authentication used ground truth, change the relevant authentication. For now, whenever a route checks if a user is an admin, just do if (true), and in the comments, add a TODO: add admin authentication, as this will be handled later when permissions are added into the api.

Please make sure to test the endpoints thoroughly to ensure they work as intended, and change the error handling to match how it works in the api repo.

I know this is a pretty large ticket, but I hope this gives you the opportunity to explore more backend tasks and how they work in our repos.

Update team data request

Develop a PUT route that can be called to make changes to a team's name, description and public status. A route as so:

update team (put "/teams/update").

• pass in the above data along with a team ID and update the team with the data.
• not all parameters may be passed in for a given request

Where will this likely be used:

• updating the team after modifying the team profile (changes could occur for any of the stats above)

Add handling for uploading a resume through registration

More team management

get team (get "/teams/:id")

• get info for one team by request param id

get teams for user (get "/teams/user/:userId")

• get teams for a user by userId params
• should also have optional search param to filter the teams by a certain event

Condense firebase-users api routes into general users route

Currently, we have two sets of "users" routes that pull information for users. One of them, firebase-users pull information about users directly from firebase. The other route pulls user information from MongoDB through the profiles table. This information can be condensed into one by pulling the user firebase information through the userId whenever a profile is retrieved.

As such, use the Firebase admin auth APIs to retrieve user information and add it onto the users api service.

Add more user roles

To see if they're a current member, on hiatus, previous member etc.

Create company schema

For our events, we need to keep track of companies and which user's are part of which company. For example, sponsor people that come to our events will be part of a certain company, and we use this information to track and give them access to data. For this reason, create the Company object type in the users service

Company

• name - string, required
• defaultEmailDomains[], string[], default: [], has a list of domains that if a user is created, they will be auto added to this company
• hasResumeAccess, boolean, default false, determines if this company can access resumes
• employees[], string[], default[], has an array of userIds who are employees at this company

additionally, create these routes in a new file

• POST /companies - create company
• GET /companies/:id - get info about a company
• PUT /companies/:id - set all info for companies
• POST /companies/:id/employees/add - post body contains emails of users to add (separated by commas), lookup each person’s uid by email with firebase auth
• PUT /companies/:id/employees - set employees array based on past body

Add cms text content to hexathons service

In the hexathons service, create a new model block for any information we need for our event sites that was originally pulled from cms.

Block Schema

{
    id: string
    hexathonId: string
    title: string
    slug (shortName): string [how we group blocks together]
    content: string (markdown)
}

Also create these relevant endpoints

GET /: Get all the blocks [required parameter for hexathon with an optional additional query parameter for slug]
POST /: Create a new block
GET /{id}: Get a specific block by id
PUT /{id}: Edit an existing block

Integrate expo

• Move over timber/expo's endpoints (see open issues as well) - look here
• Update docs

Integrate piranha

• Move over piranha's endpoints (see open issues as well) - look here
• Update docs

Create a management UI for hexathons

Create a UI on the admin page to manage hexathons since right now it can only be updated through endpoints. Also, this UI should be able to update the hexathon logo and email header and upload these to the hexlabs-public-cdn Google Cloud bucket.

Integrate in application grader api into registration service

Based on the application bot repo, integrate in the endpoints and the schema into the registration service. First, take a look through the repo and try to understand what's going on. The schema file is here and this the applications are graded based on the criteria and the scores the users give them. Then, try and integrate in the routes from the application routes file to grade applications. Try and adapt the routes so they follow the REST api format we've been doing, and integrate in with the applications schema we have.

Don't worry about the bot gradings for now, as that stuff will be integrated in later. Additionally, there is the concept of calibration questions, as these initial questions are the same for each user. Based on how each user grades these responses, we skew each user for their bias.

Additionally, you can keep the config files as they have, and we will eventually move those over to the Mongo database.

Overall, try and understand the code first, and let me know as you have any questions!

Add delete user routes to integrate with the improved login/users

In the users service, add the functionality to be able to delete a user from the database (add protection so that only admins are able to do this). This will allow us to better assist participants who may have registered under multiple emails.

A branch has already been created for this issue called feat/modify-user-endpoints

git fetch origin
git checkout feat/modify-user-endpoints

Add events service model and endpoints

For the events service (in /services/event), this service will handle our different events we throw (eg. HackGT, Horizons, etc.). This service should help manage these events, and allow us to create new ones and edit our existing one.

For this task, create the mongoose model for an event, and fill in the corresponding routes. For the routes, only fill in the first 2.

For the model, here are the fields we want to include:

• name (type is string, and required)
• isActive (type is boolean, and required, default is true, basically determines if an event is ongoing or ended)

For the routes, only fill in the first 2:

• The first route should return a list of all the events
• The second route should create a new event based on the POST body data

Create team management request

Modify the team schema to add a new field called

• memberRequests, { userId: (string, required), message: (string, optional) }[], default[]

Basically, whenever someone wants to join a team, they can send a request to join a team with an optional message

For this issue, modify the POST /teams/join route to instead make a new user on the memberRequests. This route should handle the following cases:

• If a user already has a pending request, it should remove the pending member request on a another team

Also, we should create a new route called POST /teams/:id/accept-user, where the user who makes the request should be a person on the team. Then, in the body they pass in the userId of the person they want to accept.

Create profile search

On this profiles GET route, add multiple search params to be able to search for users. You should be able to search by user's names (first, middle, last) and phone number. You should also include pagination.

Here are the query params to include

• search, this should be a text field with search stuff
• limit shows how many valid objects to return
• offset shows how much to offset by in pagination
• regex = true if its enabled, decides whether the search term uses regex or not

Your return should include these fields

• offset shows the current offset
• total total number of objects that match
• count number of objects returned in this query
• profiles actual profile objects of users

You should be able to use mongoose tools to skip, search, and query by regex. You can look here and here for some inspiration and ideas.

Add profile service model and endpoints

For the profile service (in /services/profile), this service will handle creating a profile for a user. This service will handle the personal information for a user, and managing that as well

For this task, create the mongoose model for a profile, and fill in the corresponding routes. For the routes, only fill in the first 2.

For the model, here are the fields we want to include:

• name
  • first (type: string, required)
  • middle (type: string, optional)
  • last (type: string, required)
• phoneNumber (type: string, optional)
• gender (type: string, optional)

For the routes, only fill in the first 2:

• The first route should return a list of all the profiles
• The second route should create a new profile based on the POST body data

Add resume field to profile

Every user should have the option to upload their resume to be shared with sponsors as needed. For this issue, we'll be dealing with the first instances of cross service requests in the microservices architecture.

The two services you'll be working with are the users service and the files. We'll be trying to create a link between storing a resume for a user, and having the files service store and upload their resume. You can make calls between services using the apiCall method that is in the /common folder.

For the first part, add an optional field called resume to the user model in the users service. This field should be of type ObjectId as it will store a reference to the files object.

Then, edit the files object to add a new enum called type. For now, you can use resume and other as the types for the enum.

Then, edit the files routes, specifically the /upload route. If the req.body.type field is resume, then the file should be created with this type. Additionally, you should make a cross-service PUT request to update the user profile with the corresponding id and add the resume object id as the resume.

In the end, you should be able to make a file upload request to files, and then the profile should be corresponding updated with the file object.

Create user permission groups

In the current app, we don't currently have a way to distinguish between user permission groups, like who's a member of our organization, who's on exec, etc. As such, this issue deals with adding in user permissions.

On the user profile model, add a new field called permissions with the following types

• permissions
  • member (boolean, default false, specifies whether they're a member of the org)
  • exec (boolean, default false, specifies whether they're on exec)
  • admin (boolean, default false, specifies whether they are an admin who has access to tech products)

Additionally, change the user profile routes here to account for these new permissions. Modify the / POST route so that instead of ...req.body it specifies all the fields to create except permissions. This prevents someone from setting their own permissions.

Same thing for the /:userId route, except in this case, if the req.user is an exec member or admin, they are allowed to change permissions otherwise these should not be changed on the underlying data object.

Finally, update the /GET route to allow filtering based on user permissions through request queries. For example, if ?member=true&admin=false, this updated route filter should be able to handle that. Here is an example on how to write filter queries.

Bring over email notifications service from buzzer

Looking at our old notifications repo called buzzer, bring over the email sending backend code to the notifications service. This email sending code should use Sendgrid to send emails to the specified people in the request body.

In the buzzer code, look at EmailConfig for an idea of how the configuration should be templated. For the other process.env variables, add them into the new api repo in the CommonConfig section in the config folder. For example like process.env.MAIL_TWITTER_HANDLE.

Create one new post endpoint in the notifications service called /email/send with with the post data sent to this endpoint, will send an email message. By the end, this should be a working endpoint. You can also copy over the email-template code and use this.

Update registration branch schema and validation

On the frontend, @yoshikikakehi implemented a playground using the library react-jsonschema-form. This library uses two json templates, JSONSchema and UISchema to autogenerate forms for users. We are gonna use this tool to create our registration forms based on JSONSchemas. The first one JSONSchema specifies the format of the data and which fields to include. The second JSON, UISchema specifies how the form is displayed, like autofocus, and default values.

Your job is to implement this saving and validation on the backend side, to ensure that the user's application data follows the format we've specified.

First, add the following fields to the branch model in the registration service.

• schema
  • jsonSchema (type: Object, required: true, default {})
  • uiSchema (type: Object, required: true, default {})

Then, update the branch routes in the routes folder to account for this update.

After this, this library uses Ajv JSON schema validator draft-07 to validate the code. Implement this on the backend using the getting started as a helper function that takes in the data to validator, and the branch to validate against. Make sure you test out the function and include it in the application routes to validate the applicationData as it comes in.

Create date fields for hexathon

For the hexathons service and hexathon model, there is not current way to represent the start and end date for the event. BTW, a "hexathon" is our generic name for an event, as it encompasses hackathon, makeathon, etc. So modify the hexathon model to be able to create and store a start and end date field, and then update the according routes and the docs site through the swagger-output.json file to reflect the changes.

Update registration models

For our registration, we want to have more specific fields regarding what our application data is like. As such, add these fields to the application model under the registration service. Change the field applicationData from Mixed, to having these fields:

• adult (boolean, determines if they're 18 or older)
• occupation (string, enum, student or worker, determines if theyre a student or worker)
• school (string, school of person)
• graduationYear (number)
• major (string)
• shirtSize (string)
• dietaryRestrictions (string)
• phoneNumber (string)
• gender (string)
• ethnicity (string
• Address
  • line1
  • line2
  • city
  • state
  • zip
  • country
• website
• linkedIn

Additionally, to handle other info that can vary per hackathon, we have these fields

• extraInfo (is an array of objects, can be any other questions thats determined per hackathon)
• confirmChecks (an object, that has strings to booleans, has checks like "agreesToWaiver: true", "consentsToPhotos: true")
• essays (an array of objects, that contains extra essay info for each applicant)

Based on these changes, update the application routes to handle this case. The input format for the / POST route and the /:id PATCH route should look like this:

• applicationStatus (string, required, either DRAFT or SUBMIT, determines if the application is being submitted or not)
• applicationData (an object containing all of the data as listed above)
• hexathon (an id field containing the id of the hexathon)
• branch (an id field containing the id of the branch to apply to)

Based on these fields, create/update an application. If it's a new application being submitted, set the applicationStartTime field. And then, only if the applicationStatus is SUBMIT, set the applicationSubmitTime field.

Then, create a new route to handle the confirmation, as a POST to /:id/confirmation. This data input should look like this:

• confirmationData
• hexathon

Based on this, set the confirmationStartTime and confirmationSubmitTime which will be the same value, as we won't allow them to save the status as a draft.

Add file upload

In the new files service, add the ability to view all files, get a file, upload a new file. For this, we will use Google Cloud Storage buckets to upload and store the file in the bucket called hexlabs-api-files. Make sure all the work you do is in the hexlabs-cloud project.

References for help

• File upload example in piranha for receipts
• Insight resume upload
• Registration resume file

For the file storage schema, it should store the following:

• userId for user who uploaded file (required)
• mimetype for type of file (required)
• fileName for name of file (required)
• storageId (or something similar to store the id for google retrieval)

By the end, you should be able to upload a file, get a file download link based on the file id, and also view the files in your browser with a signed url.

Feel free to consult online tutorials for this.

Create API documentation

We want to try setting up API documentation with OpenAPI so people who use our API know the format of the endpoints and what type of data they need to send and what data they'll receive.

Here are some links to look into

Generating swagger api docs

Swagger autogen

express open api

swagger ui express

An example of the type of documentation we are going for is like the stripe documentation where you can see the description of all the endpoints, and view sample responses as well. For this issue though, setup the basic framework for documentation.

This documentation site will be hosted separately from the main api, so you will have to create a new web server in another folder.

Finish events and profiles routes

In the events and profiles services, finish up the 2 remaining routes in each service. The 3rd one should get an event/profile by the request id param. The last one should update the corresponding object by it's id.

Add new fields to profile schema

Add new fields to support team formation repo

• job, string, optional (would be like student, working professional)
• school: {
  • name, string, optional (would be what university they're at)
  • year, string, optional (what year in school)
  • major, string, optional (what major you are)
    }
• skills, string[], optional (list of skills for this person)
• bio, string, optional (biography for person for team formation)

Switch events service to hexathon

After some discussion, it's a little confusing to have both events (refers to horizons, hackgt, catalyst) and schedule events (like workshops, mini-events, ceremonies) both be called events. Initially, we were gonna refer to the first category as "hackathons", but that doesn't really describe all of our events (like horizons is a makeathon, catalyst is a learnathon).

Thus, we decided to refer to these terms as "hexathon". This way, we can cover all the different types of "-thons" we have. As such, the current events service we have needs to be switched over to a "hexathon" service.

As such, rename and move all the references to the events service to hexathon service.

Add branch routes

For the branch model you just created, fill in the 4 branch routes in this file. The routes should be as following:

• The first get routes should get a list of all branches
• The second route should create a new branch
• The third route should get a branch by id
• The last route should update a branch from the post body based on the id

Add direct adding by email for team.ts

Create a new post endpoint in team.ts that takes in an email as a parameter and queries for the specific user and adds them to the team the user that pinged the endpoint is a part of.

To summarize the use case:

• user adds an email
• endpoint is called
• user associated with email is found
• above user is added to team of the user that added the email
• return the updated member array

Add in safety checks for registration

On the registration service, there are multiple safety checks we need to handle.

First, a certain registration branch is only open for certain period denoted by open and close on the model. As such, use the node date-time library called luxon to handle this checking. For all the pertinent application and confirmation routes, ensure that a user is only submitting/updating an application within the valid times, otherwise throw an error.

If a user has already submitted an application for a certain branch, they can't resubmit or edit that application. So, if a user tries to either submit a new application or edit an application after they have submitted (this is denoted by applicationSubmitTime having a value there should be an error thrown.

Additionally, if user tries to submit to a confirmation branch but doesn't have a confirmationBranch value assigned to them, then an error should be thrown, as a user can't select their confirmation branch, this is decided by the admins when we accept them or not.

Additionally, when we set the open and close time for a branch in the branch routes, add checks to ensure that the close time occurs after the open time, otherwise throw an error.

Setup new interactions service

Setup a new service in the /services folder called interactions. This service will keep track of all the user interactions when they attend a schedule event (like when they attend a workshop, or ceremony). Look at the other services for a template, and you can reuse a lot of the code, but be careful what you're copying. Make sure you change the existing package.json, and updating the existing code in the gateway and config folders. Make sure the server starts properly by updating the main index.ts file.

Let me know if you have any questions!

Create database models for notifications

Currently, for the notifications service, there is no record of which messages were sent out on which date. As such, for this issue, please create two new database models for email and text messages. Each of the two routes, email and text should have two routes, one for getting all the documents in the database (GET /), and one route for sending a message (POST /send) that will create a database entry (or multiple) for each recipient.

The database formats should store the following information, but will be different for each messaging type

• the user who sent the message
• the text of the email or message of the text message
• subject of email
• where the message was sent from (like from: email or sending number for text)
• who the recipient was as far as their email or phone number
• success - boolean value of whether or not the message was sent successfully
• errorMessage - error message if notification failed to send

Create api call to add user data to applications

Blocking HackGT/registration2#11

Currently, when returning application data from the registration service, there is no identifying user information besides userId. As such, we want to include user profile data like name, email, etc. To do this, we will need to make a cross service call to the users service to retrieve this. You can do this by looking at the docs site to see which endpoint you need to hit and using the apiCall function from the common folder.

Here is a sample call you can use to help base your code that I made during the meeting (note it's not exactly right, note the red underlines)

Note, for the /applications route that retrieves multiple applications, you'll need to use the batch endpoint here to get multiple profiles at once, and then include it for each application. For the /applications/{id} route, you can just use this route.

Add permissions checking

After the initial permissions code is setup, create a handler middleware that can be used around the code to check for permissions.

Move over phone messages from buzzer

Looking at our old notifications repo called buzzer, bring over the text message sending backend code via Twilio to the notifications service. This service uses Twilio to send messages

In the buzzer code, look at TwilioConfig for an idea of how the configuration should be templated. For now, ignore the concept of groups and integrations with registration, just look at how to input a list of numbers, and how to send a message to all of those.

Create one new post endpoint in the notifications service called /text/send with with the post data sent to this endpoint, will send a text message. By the end, this should be a working endpoint.

Registration initial statistics

Based on the old version of registration statistics, we want to create a new backend route to provide initial statistics data on our registration service. See the following 3 screenshots for the 3 groups of statistics to create. All the statistics data should be returned in one object

First, these are general statistics. Create them as following

• general
  • applicationStarted (list number of users with a started application who haven't submitted)
  • applicationSubmitted (lists number of users with a submitted application)
  • confirmed (lists number of users who have a confirmation branch assigned)

---

• applications
• for each branch thats of type application, list the name of the branch as the key, and the value is the number of applications

---

• confirmations
• for each branch thats of type confirmation, list the name of the branch as the key, and the value is the number of confirmations

Create endpoint to compile extra information of all applications

Upon request from the HackGT Emerging Committee, create an endpoint that compiles the extraInfo field of all applications.

Add checkin service model and endpoints

For the checkin service (in /services/checkin), this service will handle checking in a user for an event. This service will handle solely checking a person in to an event at the very beginning.

For this task, create the mongoose model for a checkin, and fill in the corresponding routes. For the routes, only fill in the first 2.

For the model, here are the fields we want to include:

• userId (type: string, required. this can be retrieved from req.user
• eventId (type: mongoDb Object Id, optional, this will be linked to another collection storing events. For now, leave it blank)
• status (type: string, default should be NOT_CHECKED_IN)

For the routes, only fill in the first 2:

• The first route should return a list of all the check in statuses
• The second route should create a new checkin based on the POST body data

Create registration application model and routes

For the registration service (in /services/registration), this service will handle routes and services related to registration. This service will handle creating new branches, applications, etc.

For this task, create the mongoose model for an application in the appropriate folder and create a new model file called application.ts

For the application model, here are the fields we want to include:

• user (type: string, required: true)

• event (type: objectid, required: true)

• applicationBranch (type: objectId, required)

• applicationData (type: mixed) (this will have the data for this application, and the data can vary for different events, so we used the mixed type)

• applicationStartTime (type: date, required)

• applicationSubmitTime (type: date)

• confirmationBranch (type: objectId)

• confirmationData(type: mixed) (similar reasoning to applicationData)

• confirmationStartTime (type: date)

• confirmationSubmitTime (type: date)

For the following, look at checkin.ts schema to see how I added this in automatically

• updatedAt (type:date)
• createdAt (type: date)

Make sure to create both the typescript schema and the mongoose schema

For the routes, create the following

• GET route at / : should return a list of all applications
  • Should have a way to filter by event based on query param
• GET route at /:id get a specific application by id
• POST route at / to create a new application
• PATCH route at /:id to edit an existing application