Skip to content

Sentry improvements#241

Draft
rosemcc wants to merge 2062 commits intomasterfrom
sentry
Draft

Sentry improvements#241
rosemcc wants to merge 2062 commits intomasterfrom
sentry

Conversation

@rosemcc
Copy link
Copy Markdown
Contributor

@rosemcc rosemcc commented Feb 28, 2022

Description

This PR adds a couple of improvements for Sentry tracking, including adding sourcemaps to releases via the CI/CD pipeline, and adding user IPs to enable auto session tracking (need to test that this works - documentation is unclear)

Solution

  • add to sentry initialisation config: initialScope: { user: {ip_address: "{{auto}}"} }, autoSessionTracking: true

  • Modify angular.json to generate sourcemaps in 'hidden' mode, i.e. "sourceMap": { "hidden": true,

  • Modify the Jenkins pipeline to include the following steps:

    • set the build version as the commit SHA, so it can be used as the Sentry release number
    • set the Sentry environment variables (auth token, project name etc)
    • create a sentry Release using the commit SHA and upload the sourcemaps as part of the release
  • Remove the current Sentry release workflow from the .github folder.

Screenshots

N/A

Testing

N/A

Have the changes been checked in the following browsers?

N/A

rosemcc and others added 30 commits October 12, 2021 11:18
…b-search-proxy/serverless-domain-manager-5.1.5

Bump serverless-domain-manager from 4.2.3 to 5.1.5 in /hub-search-proxy
commit 0c37b5c
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:31:14 2021 +1300

    fixed single quotes

commit 1948aaa
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:12:24 2021 +1300

    removed subhub list from subhub component

commit e84ef63
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:11:23 2021 +1300

    removed software list from software component

commit 7e83076
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:09:07 2021 +1300

    removed service list from service component

commit c53483f
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:07:22 2021 +1300

    removed funding list from funding component

commit 13804e0
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:04:12 2021 +1300

    removed event list from event component

commit d72ba54
Author: Luke <[email protected]>
Date:   Thu Oct 14 15:00:24 2021 +1300

    removed all equipment query

commit aa61a3c
Author: Luke <[email protected]>
Date:   Thu Oct 14 14:53:26 2021 +1300

    removed equipment list from equipment component

commit e68a378
Author: Luke <[email protected]>
Date:   Thu Oct 14 14:49:28 2021 +1300

    removed case study list from case study component

commit e5b6ce5
Author: Luke <[email protected]>
Date:   Thu Oct 14 14:43:57 2021 +1300

    removed articles list from article component

commit a96a5c5
Author: Luke <[email protected]>
Date:   Thu Oct 14 14:19:33 2021 +1300

    broke out subhub list

commit e39123f
Author: Luke <[email protected]>
Date:   Thu Oct 14 13:46:42 2021 +1300

    broke out software list

commit 49bc11c
Author: Luke <[email protected]>
Date:   Thu Oct 14 13:33:38 2021 +1300

    broke out service list

commit d644bf7
Author: Luke <[email protected]>
Date:   Thu Oct 14 13:04:01 2021 +1300

    changed import for funding route

commit 31885cc
Author: Luke <[email protected]>
Date:   Thu Oct 14 12:35:12 2021 +1300

    broke out funding list

commit c1fd1d5
Author: Luke <[email protected]>
Date:   Thu Oct 14 11:48:14 2021 +1300

    broke out event list

commit 8a72795
Author: Luke <[email protected]>
Date:   Thu Oct 14 11:28:38 2021 +1300

    broke out equipment list

commit 69b7d72
Author: Luke <[email protected]>
Date:   Thu Oct 14 11:17:19 2021 +1300

    broke out case study list

commit 9906d3f
Author: Luke <[email protected]>
Date:   Thu Oct 14 11:06:45 2021 +1300

    unsubscribe on destroy

commit faecf67
Merge: a21e952 baf782a
Author: Luke <[email protected]>
Date:   Thu Oct 14 10:32:34 2021 +1300

    Merge branch 'master' into build-new-content-types-list-page

commit a21e952
Author: Luke <[email protected]>
Date:   Thu Oct 14 10:32:12 2021 +1300

    broke out article list
* version update

* fix embedded asset block file size display

* correct tag manager script in the prod build

* version bump
rosemcc and others added 24 commits January 18, 2022 10:19
…b-search-proxy/contentful-export-7.14.11

Bump contentful-export from 7.13.34 to 7.14.11 in /hub-search-proxy
Bumps [nodemon](https://github.com/remy/nodemon) from 2.0.12 to 2.0.15.
- [Release notes](https://github.com/remy/nodemon/releases)
- [Commits](remy/nodemon@v2.0.12...v2.0.15)

---
updated-dependencies:
- dependency-name: nodemon
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
…r-graphql/nodemon-2.0.15

Bump nodemon from 2.0.12 to 2.0.15 in /cer-graphql
* added animation to notification

* 1st version remember persistant notification

* refactored notification request into service

* formatting

* code cleanup and comments

* minor fix to method description

* more changes to comments

* unit tests for getNotification() function

* removed some unused code from unit test

* added test for storeCurrentNotificationVersion

* improved close function

* removed focused describe

Co-authored-by: Rose <[email protected]>
* remove unused dependencies and use lockfile version 2

* update sentry to latest

* update rxjs, npm audit fix

* fix interface change to iif() in rxjs 7

* fix chunkload error

Co-authored-by: rosemcc <[email protected]>
* update ng-mocks

* update @angular/cli

* update @angular/material @angular/core

* npm update

* removed workaround for mat-card-image on picture tags

* added missing mat-card-image directive

* fixed default images not rendered properly, minor re-factor

* minor tidy up

* update karma version
* ng add @angular/pwa

* set theme-color

* replace default Angular icons with UoA

* clean up codeCoverageExclude

* readme

* Material icons and fonts caching

* Revert "Material icons and fonts caching"

This reverts commit 687a52a.

* material icons, uoa fonts, contentful assets caching, readme

* lazy fetch contentful assets

* service worker on prod index.html

* App name

* add script to run in service worker mode locally

* csp update WIP

* 504 gateway timeout bypass

* sw updates service WIP

* note

* unit tests passing

* more csp updates due to service worker fetching

* about link fix on footer

* adding sw-updates functionality

* add nontransparent icon for apple-touch-icon

* replace deprecated methods, add error handling

* package-lock fix after merge

* tidying, add types

* material dialog for update confirmation

* tests passing

* Change dialog to snackbar, add styling

* add a test

* remove unused material modules

* button style

* minor update to script and readme for running service worker locally

* fix snackbar style

* add app id to manifest

* refactor swUpdates, add tests

* remove f

* add maskable icons

* icons fix

* add apple touch icon

* changes to pipe and unit test

* remove unused imports

Co-authored-by: Luke <[email protected]>
Bumps [contentful-export](https://github.com/contentful/contentful-export) from 7.14.11 to 7.14.27.
- [Release notes](https://github.com/contentful/contentful-export/releases)
- [Commits](contentful/contentful-export@v7.14.11...v7.14.27)

---
updated-dependencies:
- dependency-name: contentful-export
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
…b-search-proxy/contentful-export-7.14.27

Bump contentful-export from 7.14.11 to 7.14.27 in /hub-search-proxy
…search-hub-web/graphql-codegen/typescript-2.4.3

Bump @graphql-codegen/typescript from 1.23.0 to 2.4.3 in /research-hub-web
@rosemcc rosemcc requested a review from eric-el-tan March 18, 2022 01:01
export const environment = {
env: 'prod',
sentryTracesSampleRate: 0.2,
version: 'VERSION',
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testing a comment

@rosemcc rosemcc removed the request for review from eric-el-tan March 21, 2022 20:35
Comment on lines +15 to +36
name: Run linters
runs-on: ubuntu-latest

steps:
- name: Check out Git repository
uses: actions/checkout@v2

- name: Set up Node.js
uses: actions/setup-node@v1
with:
node-version: 14

- name: Install Node.js dependencies
working-directory: ./research-hub-web
run: npm ci

- name: Install Angular CLI
run: npm install -g @angular/cli

- name: ng lint
working-directory: ./research-hub-web
run: ng lint

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 days ago

To fix the problem, explicitly restrict GITHUB_TOKEN permissions in the workflow to the minimal scope required. This workflow only needs to read repository contents to check out and lint the code, so contents: read is sufficient. We can set this at the workflow (root) level so it applies to all jobs, or at the job level. Since there is only one job, either is fine, but using a top‑level permissions block best documents the workflow’s intent and secures any future jobs by default.

Concretely, in .github/workflows/linting.yml, add a permissions: block near the top, at the same indentation level as on: and jobs:. For example:

name: Lint

on:
  ...

permissions:
  contents: read

jobs:
  ...

No imports or external libraries are needed; this is purely a YAML configuration change within the workflow file and does not alter the existing functional behavior of the linting steps.

Suggested changeset 1
.github/workflows/linting.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/linting.yml b/.github/workflows/linting.yml
--- a/.github/workflows/linting.yml
+++ b/.github/workflows/linting.yml
@@ -10,6 +10,9 @@
     branches:
       - master
 
+permissions:
+  contents: read
+
 jobs:
   run-linters:
     name: Run linters
EOF
@@ -10,6 +10,9 @@
branches:
- master

permissions:
contents: read

jobs:
run-linters:
name: Run linters
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines +14 to +33
name: Create Sentry Release
runs-on: ubuntu-latest

steps:
- name: Check out Git repository
uses: actions/checkout@v2
- name: Get Branch
id: var
run: echo ::set-output name=branch::${GITHUB_REF#refs/*/}
- name: Output Branch
run: echo ${{ steps.var.outputs.branch }}
- name: Notify Sentry
# https://github.com/getsentry/action-release
uses: getsentry/[email protected]
env:
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_ORG: university-of-auckland-7o
SENTRY_PROJECT: research-hub
with:
environment: ${{ steps.var.outputs.branch }} No newline at end of file

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 3 days ago

In general, fix this by explicitly configuring permissions for the workflow or for the sentry-release job so that the GITHUB_TOKEN has the minimal scope needed. For this workflow, the steps only need to read the repository (for actions/checkout); the Sentry release uses its own auth token, so no GitHub write scopes are required. Therefore, adding permissions: contents: read at the workflow or job level is sufficient and preserves existing behavior.

The best minimal fix without changing functionality is to add a permissions block at the job level under jobs.sentry-release. Concretely, in .github/workflows/sentry.yml, under line 14 (name: Create Sentry Release) and aligned with runs-on, add:

permissions:
  contents: read

No additional methods, imports, or other definitions are required; this is purely a configuration hardening change to the workflow YAML.

Suggested changeset 1
.github/workflows/sentry.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/sentry.yml b/.github/workflows/sentry.yml
--- a/.github/workflows/sentry.yml
+++ b/.github/workflows/sentry.yml
@@ -13,6 +13,8 @@
   sentry-release:
     name: Create Sentry Release
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
 
     steps:
       - name: Check out Git repository
EOF
@@ -13,6 +13,8 @@
sentry-release:
name: Create Sentry Release
runs-on: ubuntu-latest
permissions:
contents: read

steps:
- name: Check out Git repository
Copilot is powered by AI and may make mistakes. Always verify output.
run: echo ${{ steps.var.outputs.branch }}
- name: Notify Sentry
# https://github.com/getsentry/action-release
uses: getsentry/[email protected]

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Sentry Release' step
Uses Step
uses 'getsentry/action-release' with ref 'v1.1.6', not a pinned commit hash

module.exports.search = async (event, context) => {
try {
console.log(`Received query: ${event.body}`);

Check warning

Code scanning / CodeQL

Log injection Medium

Log entry depends on a
user-provided value
.

Copilot Autofix

AI 3 days ago

To fix the problem, user-controlled input (event.body) should be sanitized before being logged. For plain-text logs, the main risk is newline and carriage-return characters that let an attacker forge extra lines; these should be stripped or replaced. It is also helpful to clearly mark user input in the log message so it is visually distinguishable from surrounding text.

The best minimal fix without changing functionality is to introduce a small helper that sanitizes strings for logging by removing \r and \n, and then use it when logging event.body. This keeps everything else the same (we still log the request body, and the search behavior is unchanged) but prevents log injection. We can define a simple sanitizeForLog function near the top of handler.js and then change line 53 from logging event.body directly to logging a sanitized version. No existing imports need to be changed; the helper uses only built-in string operations.

Concretely:

  • In hub-search-proxy/handler.js, add a function such as:
    function sanitizeForLog(value) {
      if (typeof value !== 'string') {
        try {
          value = JSON.stringify(value);
        } catch (e) {
          value = String(value);
        }
      }
      return value.replace(/[\r\n]+/g, ' ');
    }
  • Replace the console.log at line 53 with:
    console.log(`Received query: ${sanitizeForLog(event.body)}`);

This ensures that any newline or carriage-return characters in the potentially user-controlled event.body are removed (or collapsed to spaces) before being logged, mitigating log injection while preserving the rest of the behavior.

Suggested changeset 1
hub-search-proxy/handler.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/hub-search-proxy/handler.js b/hub-search-proxy/handler.js
--- a/hub-search-proxy/handler.js
+++ b/hub-search-proxy/handler.js
@@ -20,6 +20,17 @@
 
 const VALID_CONTENT_TYPES = ['article','casestudy','equipment','event', 'funding', 'service','software','subhub'];
 
+function sanitizeForLog(value) {
+  if (typeof value !== 'string') {
+    try {
+      value = JSON.stringify(value);
+    } catch (e) {
+      value = String(value);
+    }
+  }
+  return value.replace(/[\r\n]+/g, ' ');
+}
+
 let credentials;
 try {
   // try getting credentials for the lambda from the AWS environment
@@ -50,7 +61,7 @@
 
 module.exports.search = async (event, context) => {
   try {
-    console.log(`Received query: ${event.body}`);
+    console.log(`Received query: ${sanitizeForLog(event.body)}`);
     const requestBody = JSON.parse(event.body);
     let queryString = '';
     let size = 10;
EOF
@@ -20,6 +20,17 @@

const VALID_CONTENT_TYPES = ['article','casestudy','equipment','event', 'funding', 'service','software','subhub'];

function sanitizeForLog(value) {
if (typeof value !== 'string') {
try {
value = JSON.stringify(value);
} catch (e) {
value = String(value);
}
}
return value.replace(/[\r\n]+/g, ' ');
}

let credentials;
try {
// try getting credentials for the lambda from the AWS environment
@@ -50,7 +61,7 @@

module.exports.search = async (event, context) => {
try {
console.log(`Received query: ${event.body}`);
console.log(`Received query: ${sanitizeForLog(event.body)}`);
const requestBody = JSON.parse(event.body);
let queryString = '';
let size = 10;
Copilot is powered by AI and may make mistakes. Always verify output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants