Coder Social home page Coder Social logo

cal-itp / benefits Goto Github PK

View Code? Open in Web Editor NEW
22.0 22.0 9.0 15.03 MB

Transit benefits enrollment, minus the paperwork.

Home Page: https://docs.calitp.org/benefits

License: GNU Affero General Public License v3.0

Dockerfile 0.28% Python 63.82% HTML 18.94% CSS 6.14% Shell 1.22% JavaScript 2.62% HCL 6.98%
django

benefits's People

Contributors

afeld avatar angela-tran avatar calexity avatar dependabot[bot] avatar indexing avatar jorden1420 avatar kylehamilton avatar machikoyasuda avatar mattrenda avatar pre-commit-ci[bot] avatar pyup-bot avatar sureshtekyentra avatar thekaveman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

benefits's Issues

Secure critical Django cookies

The following can be configured in settings.py.

Setting these ensures the cookies expire and are deleted when the user's browser closes (makes them session cookies)

We also want to prevent the CSRF cookie from being sent in cross-browser requests. E.g. a user clicks on a link to our site, from another site - in this case we don't want any previous CSRF cookie already in their browser sent. It's unlikely to be an issue in our app since there are no user logins etc. but for consistency with our other cookies (session and language).

Look at implementing HTTP Strict Transport Security

HTTP Strict Transport Security or HSTS:

For sites that should only be accessed over HTTPS, you can instruct modern browsers to refuse to connect to your domain name via an insecure connection (for a given period of time) by setting the “Strict-Transport-Security” header. This reduces your exposure to some SSL-stripping man-in-the-middle (MITM) attacks.

From https://docs.djangoproject.com/en/3.2/ref/middleware/#http-strict-transport-security. We may want/need to do this at the Nginx level instead, see https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/

Crucial to this decision is the understanding that the AWS Load Balancer Azure App Service handles SSL connections for this app, and not the Django/Nginx system; the domain's certs and HTTP -> HTTPS redirection is all managed at the AWS Load Balancer App Service level.

See for example this app's nginx.conf in which the Nginx server is listening on port 8000 for connections - this is because the Load Balancer sits between the user and Nginx, handling the SSL and translation into the app container. In which case, trying to set this from within the app container may cause more problems than it solves.

Eligibility confirmation spec issue

This spec: https://github.com/cal-itp/benefits/pull/170/files#diff-1637a357180023d868d2fca69d3d4e2a68f7b58c61aa71b016977ca95ee672fcR36 is currently set to skip() because it does not pass with the way the app is currently set up.

Previously, the Eligibility Server API had fake merchant auth token routes to accept calls from the abc transit agency in development mode, but these have since been removed - as of this PR https://github.com/cal-itp/eligibility-server/pull/31/files#diff-e5991fd559847cf92df779bd8b72643e9b64926355be788735b3c1cfcabca211L219

Now, developers cannot get the eligibility/confirm page to load with valid test credentials b/c the app looks for valid tokens before it loads the page. There are several ways we can address this:

  1. Configure some sort of stubbing/mocking in the Cypress test to return fake auth tokens, so that Cypress can open the eligibility/confirm page without payment buttons. (This addresses the Cypress issue but not the local dev one).
  2. Change the way Django opens this page. https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/templates/enrollment/index.html#L26 Have Django instead load the page and then asynchronously load the payment button. (This addresses both Cypress and local dev issue.)
  3. Create a fake Merchant Auth token so the page can load (and where would that code live and how would we do it?) (What we had in the past: #179)

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Docker Compose does not use the latest main image of Eligibility Server

This line in localhost/docker-compose.yml: https://github.com/cal-itp/benefits/blob/dev/.devcontainer/compose.yml#L56 does not function as we'd assumed it would. Docker Compose is not checking to make sure that the image is using the latest main tag from here: https://github.com/cal-itp/eligibility-server/pkgs/container/eligibility-server

  server:
    image: ghcr.io/cal-itp/eligibility-server:main

Locally, I had to run docker pull ghcr.io/cal-itp/eligibility-server:main before running benefits in the Dev Container to get the latest main image. I was running a main image from several updates ago, and Docker Compose wasn't updating it.

We may have to update the docker-compose.yml file with a specific versioned tag number or the specific SHA from Eligibility Server.

Autocomplete on Eligibility form could expose user data

Background

This is only an issue in a specific kind of environment; for example if this app were deployed on a public terminal (e.g. at a library or a transit agency customer service center).

Any time a user interacts with the Eligibility form, they could potentially see entries from other users who previously interacted with the form, whether or not those prior interactions were successful validations.

Remediation

We can attempt to turn off autocomplete on form input elements, if this is deemed a big enough risk.

However, it should be noted that this is not as straightforward as it may seem, see for example these SO threads:

It sounds like Chrome in particular does a lot of work to undo choices web developers make and turn autocomplete back on.

Prevent ELB healthcheck from logging Amplitude events

I thought this was fixed as part of #86 when we implemented a specific Django route/view to respond to /healthcheck requests.

But apparently not, because Amplitude is full of ELB healthcheck events:

image

Drilling in to any of these shows similar details:

image

Automate cypress runs for PRs

#122 introduced an initial Integration/End-to-End test setup, but the only way to run tests right now is on localhost by hand.

We'd like to instead run the tests on each PR as a merge check in GitHub Actions. Similar to the work over in cal-itp/eligibility-server#16.

Basic outline is:

Rider MST CC information successfully verified

  • As a rider with a valid MST Courtesy Card, I want to share my program enrollment information and see that I am eligible to participate in the contactless fare-discount demonstration.
  • As a rider with an invalid MST Courtesy Card, when I enter in my program enrollment information, I should get a screen that says my eligibility could not be confirmed.

Acceptance Criteria

  • Rider can view a form to enter their last name and MST Courtesy Card number.
  • Rider can enter their last name and MST Courtesy Card number (last name must be the same case)
  • When rider inputs the correct credentials (card number and last name), and clicks “check status”, the rider is shown a “Checking” status in the status button.
  • Rider information is successfully verified in the dataset
  • Rider is shown a success screen

Resources

Add MST Courtesy Card Option and Document Info

As a rider with a MST Courtesy Card, I want the option to sign up with a MST Courtesy Card so I can enroll in the contactless fare-discount demonstration.

Acceptance Criteria

  • Riders can select senior or MST CC discount options.
  • When a rider selects “Senior Discount Program ” and clicks “continue” they will be redirected to the senior discount document page
  • When a rider selects “MST Courtesy Cardholder” and clicks “continue” they will be redirected to the courtesy cardholder discount document page.

Resources

Enrollment page is accessible without passing eligibility verification

If a user knows the /enrollment endpoint, and has configured their session with a Transit Agency, they can navigate to the credit card enrollment page and completely bypass the EV check. This is a serious bug!

Steps to reproduce

  1. Visit https://test-benefits.calitp.org
  2. Click either of the Agency buttons
  3. Manually update the URL to https://test-benefits.calitp.org/enrollment
  4. Profit

Remediation

This page must ensure the user has previously verified eligibility criteria

Push builds into GHCR

We build a Docker image and push it to AWS for use in the production app, but this sits inside a private ECR repository.

It could be useful to have the image on a public repo as well. Based on the work from cal-itp/eligibility-server#22, we can add an additional docker/build-push-action step to our dev deployment that pushes into the GitHub Container Registry:

- name: Build and push main
  uses: docker/build-push-action@v2
  if: ${{ github.event_name != 'release' }}
  with:
    push: true
    tags: ghcr.io/${{github.repository}}:<branch>

We can do this for each of our 3 environments to always have the current image running in each environment, available in GHCR as well.

Since #135 landed, additional docker/build-push-action steps should go very fast.

VS Code Devcontainer fails to set up on Linux

Expected Result

The devcontainer is successfully set up.

Actual Result

An error is thrown when trying to run the postAttachCommand, specifically when we're trying to install Cypress.

Steps to Reproduce

  1. Have Linux as your host operating system
  2. Make sure you've built the base Docker image benefits_client:latest
  3. Open the benefits folder as a remote container in VS Code

Output

From console logging:

Running the postAttachCommand from devcontainer.json...

[117148 ms] Start: Run in container: /bin/bash localhost/bin/init.sh
[117388 ms] Port forwarding connection from 59748 > 37657 > 37657 in the container.
[117389 ms] Start: Run in container: /root/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/node -e 
pre-commit installed at .git/hooks/pre-commit
[WARNING] The 'rev' field of repo 'https://github.com/compilerla/conventional-pre-commit' appears to be a mutable reference (moving tag / branch).  Mutable references are never updated after first install and are not supported.  See https://pre-commit.com/#using-the-latest-version-for-a-repository for more details.  Hint: `pre-commit autoupdate` often fixes this.
[INFO] Initializing environment for https://github.com/compilerla/conventional-pre-commit.
...
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[119288 ms] [15:54:45] Extracted extension to /root/.vscode-server/extensions/.c2d52185-63c1-4349-ac91-40616d2fdbc8: ms-python.python
[119306 ms] [15:54:45] Renamed to /root/.vscode-server/extensions/ms-python.python-2021.10.1365161279
[119310 ms] [15:54:45] Extracting completed. ms-python.python
[119311 ms] [15:54:45] Extension installed successfully: ms-toolsai.jupyter-keymap
[119311 ms] [15:54:45] Extension installed successfully: ms-toolsai.jupyter-renderers
[119311 ms] [15:54:45] Extension installed successfully: ms-toolsai.jupyter
[15:54:45] Extension installed successfully: ms-python.python
[119316 ms] Extension 'ms-python.python' v2021.10.1365161279 was successfully installed.
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8.
[120807 ms] [15:54:46] Extracted extension to /root/.vscode-server/extensions/.1a0abef0-8ca2-4923-8aa5-ec1605a86354: ms-python.vscode-pylance
[120821 ms] [15:54:46] Renamed to /root/.vscode-server/extensions/ms-python.vscode-pylance-2021.10.3
[120823 ms] [15:54:46] Extracting completed. ms-python.vscode-pylance
[120823 ms] [15:54:46] Extension installed successfully: ms-python.vscode-pylance
[120826 ms] Extension 'ms-python.vscode-pylance' v2021.10.3 was successfully installed.
[121800 ms] Extensions cache, remote removals: None
[INFO] Initializing environment for https://github.com/pycqa/bandit.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
...
[INFO] Installing environment for https://github.com/psf/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://gitlab.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pycqa/bandit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
pre-commit installed at .git/hooks/commit-msg

up to date, audited 171 packages in 765ms

25 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Cypress cannot write to the cache directory due to file permissions

See discussion and possible solutions at
https://github.com/cypress-io/cypress/issues/1281

----------

Failed to access /root/.cache/Cypress:

EACCES: permission denied, mkdir '/root/.cache/Cypress'

----------

Platform: linux (Debian - 11)
Cypress Version: 8.7.0

Notes

@thekaveman and I speculate it has something to do with the "quirks with local filesystem (bind) mounts" with Docker CE/EE on Linux (see VS Code docs on non-root users). Windows users can see everything in the Linux container because there isn't a way to map the permissions. This comment may have some helpful details too.

There are other GitHub issues showing this error:

Refactor model to split eligibility from benefit

The EligibilityType model contains two properties used by different phases of the system:

  • EligibilityType.name is used in the eligibility phase as part of the API call to verify
  • EligibilityType.group_id is used in the enrollment phase as the group to enroll eligible users into

However, as noted and fixed in #89, we may have multiple EligiblityType with the same name and different group_id. For example, a senior type for each transit agency, each with a different group_id.

The fix in #89 was somewhat of a hack, as these concerns could (should?) be split out to allow reusing EligibilityType instances. Thinking of a refactor along the lines of:

Before

class EligibilityType(models.Model):
    """Currently used for both eligibility and enrollment"""
    name = models.TextField()
    label = models.TextField()
    group_id = models.TextField()

class TransitAgency(models.Model):
   # other fields ...
    eligibility_types = models.ManyToManyField(EligibilityType)

After

class EligibilityType(models.Model):
    """Used for eligibility checks - reused across agencies"""
    name = models.TextField()
    label = models.TextField()

class BenefitGroup(models.Model):
    """Used for enrollment - at least one per agency"""
    name = models.TextField()
    group_id = models.TextField()
    eligibility_type = models.ForeignKey(EligibilityType, on_delete=models.PROTECT)

class TransitAgency(models.Model):
   # other fields ...
   benefit_groups = models.ManyToManyField(BenefitGroup)

Version and release strategy

Since we're getting close to being live, now is a good time to implement a versioning and release strategy. This will help us talk about development and the application itself in more real terms like "These features are part of version XYZ".

Related Background

Proposal

A very rough outline of a process could look like:

  1. Merge to prod to kick off a deploy to AWS, like now
  2. After ^ deploy succeeds, increment the version number/tag using reecetech/version-increment
  3. Push the new version number/tag
  4. Make a new Release with the version number using softprops/action-gh-release (separate workflow runs when new tag shows up in GitHub)

Questions

  1. Should we include a v prefix in the version numbers? Our consensus was no, and reecetech/version-increment doesn't support this anyway. See reecetech/version-increment#8 for more.
  2. Traditional semver scheme MAJOR.MINOR.PATCH, or alternatively, "calver" scheme YEAR.MONTH.RELEASE where RELEASE is the Nth release that month. Both are supported by reecetech/version-increment.

For versioning scheme, I kind of like calver for this application, given the other reporting requirements etc. that are monthly or month-based.

Load Spanish translations for MST Courtesy Cards

  • As a spanish speaking rider with a MST Courtesy Card, I want to be able to view the tool in spanish so that I can sign up for the contactless fare-discount demonstration.

Acceptance criteria

  • All MST CC screens and Help page can be toggled into Spanish mode and accurate Spanish translations are loaded

Link credit card for valid MST Courtesy Card

  • As a rider with a MST Courtesy Card I want to provide my bank card information so that it is linked to my recently verified status as eligible for the fare-discount program.

Acceptance Criteria

  • When a rider clicks “Continue to our payment provider” on the success screen they are show the Lilttlepay credit card modal
  • Rider is able to enter a valid bank credential and select “Next”
  • Rider is shown success screen (Large_Eligibility5_Success)

Resources

Ensure cookies are enabled

Need a basic client-side check that cookies are enabled (and UI if not) as soon as the app loads, since we require cookies to store the temporary transaction data.

Figure out dynamic Docker ports for localhost

Hardcoding localhost ports to map into Docker containers is unnecessary maintenance overhead, especially when working on multiple projects each with a number of containers/services and trying to choose ports that allow all services to run simultaneously.

Ideally Docker would just pick an available port during container startup and use that. We'd like to get away from managing these values / environment variables:

  • DJANGO_LOCAL_PORT
  • DOCS_LOCAL_PORT
  • 5678 hardcoded as the debug port for dev

The following values will be moving from this project into eligibility-server as part of the work in #121, so we'll want to apply the same approach there as well:

  • 5000 hardcoded as the server port
  • 5789 hardcoded as the debug port for server

We'll need to consider:

  • Running ad-hoc docker compose ... commands
  • Running from a devcontainer with associated services

This SO post is a good starting place: https://stackoverflow.com/a/56447023/453168.

Refactor model to consolidate PEM data

  • Create a new model class:

    class PemData(models.Model):
        """An API certificate or key in PEM format."""
    
        id = models.AutoField(primary_key=True)
        text = models.TextField()
  • Run manage.py makemigrations to generate migration file, copy the new defs into 0001_initial then delete the migration file

  • Refactor EligibilityVerifier, PaymentProcessor, TransitAgency to reference the new model

  • Refactor localhost/data/client.json to the new model

  • Refactor other consumers of model class as needed

JavaScript, CSS linting and formatting

Now that we have more JavaScript in the way of Cypress tests, we should introduce some linting and formatting to the devcontainer settings and pre-commit.

Send X-XSS-Protection header

The X-XSS-Protection header can be used to manage certain browser's protection against reflected cross-site scripting (XSS), stopping a page from being loaded if an attack is detected. In modern browsers, the Content-Security-Policy header can provide better protection against XSS and setting X-XSS-Protection might be redundant (#203 tracks CSP implementation).

See more at https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection

We'll want the following header/value:

X-XSS-Protection: 1; mode=block

This can be done in a new Middleware and configured in settings.py for all requests/responses.

Production checklist

  • Create .github/workflows/ecs-deploy-prod.yml based on existing files for dev and test.
  • Configure secrets in prod environment
    • AWS_ACCESS_KEY_ID
    • AWS_ACCOUNT
    • AWS_BUCKET
    • AWS_SECRET_ACCESS_KEY
  • Add .env and config.json files to bucket
    • Verify ALLOWED_HOSTS
    • Verify amplitude API creds
    • Verify eligibility API creds
    • Verify payment processor API creds
  • Configure domain in Route53
  • Verify ELB subnets are Outside
  • Add aws-cli image to ECR
  • Merge dev -> test -> prod to initiate the deploy

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Decide how (if) to assign customer_ref

Background

Our discount provider's API includes a field customer_ref on the customer object, which can be set one time to a unique value (uniqueness as defined by the API caller, not the discount provider). This value is visible in the provider's Merchant Portal and serves as a way to correlate the customer with an outside record.

  • There is no concept of a customer database or storage of customer transactions in benefits, so there is no way to track uniqueness
  • benefits does not ask for any data beyond the basics necessary for eligibility verification, so we have no meaningful values to fill in here (e.g. email address)

Options

  1. Leave this blank
  2. Generate a value using uuid.uuid4()
  3. Correlate with e.g. an analytics ID
  4. Something else?

I lean towards 1 since we don't really have a need for this right now, and I don't believe it is a required field in the API.

//cc @vykster @calexity @krainboltgreene

Refactor eligibility verification into reusable package

Overview

We have benefits and the eligibility-server each with Python code that implements one half of the Eligibility Verification API.

We could consolidate these two disparate halves into a single Python package that would:

  • Allow for easier testing, outside the context of a web application
  • Keep the constructing and parsing of messages consistent
  • Consolidate dependency management into a single package

Note that last point is particularly relevant as we are now in the process of upgrading dependencies in the server to match recently posted updates to the same dependencies here (#131, #132).

The corollary tracking issue in server is cal-itp/eligibility-server#38

Benefits specifics

benefits/eligibility/api.py is where the client-side implementation lives.

RequestToken and ResponseToken are almost entirely independent from any Django-based context or references. Where they do make use of Django objects (e.g. a TransitAgency model is passed in) we could instead pass the needed data (e.g. the agency's ID), directly.

Client is only slightly more tied into the Django-specifics and could be easily refactored as well.

Send Content-Security-Policy header

The Content-Security-Policy (CSP) header (with the frame-ancestors directive) replaces the now deprecated X-Frame-Options header, to instruct the browser about appropriate actions to perform if a site is included inside an <iframe>.

See more at https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP

We already have Django's built-in Clickjacking/X-Frame-Options features enabled. Since this app should never be run from an <iframe>, let's create another Middleware that sets the CSP header like so:

Content-Security-Policy: default-src 'self'; frame-ancestors 'none';

Proposal: refactor local eligibility server into new repo

Background

The localhost/server directory contains a simple Flask app implementing a very basic Eligibility Verification API server. This server can be used for testing benefits with only local resources via the service defined in localhost/docker-compose.yml. Read more about running the test verification server.

Proposal

  1. Create a new repository, tentatively called cal-itp/eligibility-server (suggestions welcome)
  2. Move the current localhost/server project there
  3. Set up the usual tooling (pre-commit, mkdocs, etc.)

Goals

There are a number of advantages in having a standalone server project. A more robust implementation can help us design better testing scenarios for the client (benefits), while creating a path towards expanding into new eligibilities that may require custom verification logic. Separate projects can move at different speeds and respond to needs more nimbly.

The initial goals would be:

  • Usable by benefits in a local development context for testing (e.g. via a public Docker image)
  • Usage documentation, beyond the sparse text we have online now

During the transition, we don't need any major changes or features added for the server; the primary goal is to split the two projects apart while maintaining their current compatibility.

Alternatives

The status quo, where the server lives side-by-side in this same repository, is not all that bad. It simplifies local usage of the test server, and keeps a tight relationship between the client and server tokenization and encryption code. We could alternatively refactor the server into a top-level directory and work on it as a more monorepo concept.

Add automated Lighthouse accessibility testing for PRs

  • Add GitHub action that runs LH on the app built for the PR
  • Require Accessibility, Best Practices to pass
  • Do the Performance test but don't require it

GitHub Action:

  • Run on PRs that touch any benefits/** files
  • Build Docker image
  • Start app w/ Docker at specific port
  • Run lighthouse at that specific port

Rename main to prod

We aren't using main as the default branch. Let's rename it to prod to more closely align with the deployment environment it corresponds with.

Let's also rename the GitHub environment to prod to match.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.