Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
bba5626
Add basic support for Typescript scanning
JohT Feb 24, 2024
baf9084
Remove workaround for jQAssistant/jqa-java-plugin#44
JohT Feb 26, 2024
4e11772
Add cypher queries to explore code graph schemas
JohT Apr 9, 2024
184e24f
Enrich Graph for Typescript
JohT Mar 19, 2024
bd17dad
Make weight setter language dependent
JohT Apr 16, 2024
1398cc4
Add Object-oriented design metrics for Typescript
JohT Apr 9, 2024
a21c2e0
Make object oriented design metrics language dependent
JohT Apr 16, 2024
3bfbb82
Provide common function for returning the first non empty query resul…
JohT Apr 13, 2024
a30f847
Optimize reading or already calculated values
JohT Apr 16, 2024
11652f3
Add clean-up step after report generation
JohT Apr 16, 2024
688d7c8
Make pre report check queries language-aware
JohT Apr 17, 2024
06ba4f1
Add query function that returns the number of result lines
JohT Apr 19, 2024
f4ec8f5
Share common Graph projection functions.
JohT Apr 20, 2024
2397514
Move unused Cypher queries from the main directory to suitable sub di…
JohT Apr 21, 2024
ff97389
Exclude reports directory from linguist language detection
JohT Apr 21, 2024
917097e
Make Jupyter notebooks robust against missing data
JohT Apr 21, 2024
90ce598
Refine Wordcloud to be more universal & language agnostic
JohT Apr 21, 2024
3addb57
Use a common script for Jupyter notebook report generation
JohT Apr 21, 2024
3f21695
Clarify that existing pipeline analyzes Java code
JohT Apr 22, 2024
78881e7
Add pipeline for Typescript code analysis
JohT Apr 22, 2024
d775f62
Improve error handling on missing visualization data
JohT Apr 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,8 @@
*.tagx text
*.xml text

# These files shouldn't be taken into account for statistics
# since they are from external libraries
src/main/javascript/polyfills/** linguist-vendored
src/test/resources/jasmine/** linguist-vendored
# These generated reports shouldn't be taken into account for language statistics
results/** linguist-vendored

# These files are binary and should be left untouched
# (binary is a macro for -text -diff)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Code Structure Analysis
name: Java Code Structure Graph Analysis

on:
push:
Expand Down Expand Up @@ -50,6 +50,8 @@ jobs:
env:
CI_COMMIT_MESSAGE: Automated code structure analysis reports (CI)
CI_COMMIT_AUTHOR: ${{ github.event.repository.name }} Continuous Integration
PROJECT_NAME: AxonFramework
# Version variable name matches renovate.json configuration entry
AXON_FRAMEWORK_VERSION: 4.9.3

steps:
Expand Down Expand Up @@ -111,16 +113,16 @@ jobs:
key:
${{ runner.os }}-${{ hashFiles('**/*.sh') }}

- name: Download AxonFramework artifacts
- name: Download ${{ env.PROJECT_NAME }}-${{ env.AXON_FRAMEWORK_VERSION }}
working-directory: temp
run: |
mkdir -p AxonFramework-${{ env.AXON_FRAMEWORK_VERSION }}
cd AxonFramework-${{ env.AXON_FRAMEWORK_VERSION }}
mkdir -p ${{ env.PROJECT_NAME }}-${{ env.AXON_FRAMEWORK_VERSION }}
cd ${{ env.PROJECT_NAME }}-${{ env.AXON_FRAMEWORK_VERSION }}
echo "Working directory: $( pwd -P )"
./../../scripts/downloader/downloadAxonFramework.sh ${{ env.AXON_FRAMEWORK_VERSION }}

- name: Analyze AxonFramework
working-directory: temp/AxonFramework-${{ env.AXON_FRAMEWORK_VERSION }}
- name: Analyze ${{ env.PROJECT_NAME }}-${{ env.AXON_FRAMEWORK_VERSION }}
working-directory: temp/${{ env.PROJECT_NAME }}-${{ env.AXON_FRAMEWORK_VERSION }}
# Shell type can be skipped if jupyter notebook reports (and therefore conda) aren't needed
shell: bash -el {0}
env:
Expand Down
185 changes: 185 additions & 0 deletions .github/workflows/typescript-code-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
name: Typescript Code Structure Graph Analysis

on:
push:
branches:
- main
# Ignore changes in documentation, general configuration and reports for push events
paths-ignore:
- 'results/**'
- '**/*.md'
- '**/*.txt'
- '**/*.css'
- '**/*.html'
- '**/*.js'
- '.gitignore'
- '.gitattributes'
- 'renovate.json'
- 'changelogTemplate.mustache'
- '**.code-workspace'
pull_request:
branches:
- main
# Ignore changes in documentation, general configuration and reports for pull request events
paths-ignore:
- 'results/**'
- '**/*.md'
- '**/*.txt'
- '**/*.css'
- '**/*.html'
- '**/*.js'
- '.gitignore'
- '.gitattributes'
- 'renovate.json'
- 'changelogTemplate.mustache'
- '**.code-workspace'

# Requires the secret NEO4J_INITIAL_PASSWORD to be configured
jobs:
reports:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- os: ubuntu-latest
java: 17
python: 3.11
mambaforge: 24.3.0-0
node: 18

env:
CI_COMMIT_MESSAGE: Automated code structure analysis reports (CI)
CI_COMMIT_AUTHOR: ${{ github.event.repository.name }} Continuous Integration
PROJECT_NAME: react-router
# Version variable name matches renovate.json configuration entry
REACT_ROUTER_VERSION: 6.22.0

steps:
- name: Checkout GIT Repository
uses: actions/checkout@v4
with:
token: ${{ secrets.WORKFLOW_GIT_ACCESS_TOKEN }}

- name: Setup Java JDK ${{ matrix.java }}
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: ${{ matrix.java }}

- name: Setup node.js ${{ matrix.node }} for Graph Visualization
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}

- name: Install nodes packages for Graph Visualization
working-directory: graph-visualization
run: npm ci

- name: Setup Cache for Conda package manager Mambaforge
uses: actions/cache@v4
env:
# Increase this value to reset cache if etc/example-environment.yml has not changed
# Reference: https://github.com/conda-incubator/setup-miniconda#caching
CACHE_NUMBER: 0
with:
path: ~/conda_pkgs_dir
key:
${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-environments-${{hashFiles('**/environment.yml', '.github/workflows/*.yml') }}

# "Setup Python" can be skipped if jupyter notebook reports aren't needed
- name: Setup Python ${{ matrix.python }} with Conda package manager Mambaforge
uses: conda-incubator/setup-miniconda@v3
with:
python-version: ${{ matrix.python }}
miniforge-variant: Mambaforge
miniforge-version: ${{ matrix.mambaforge }}
use-mamba: true
activate-environment: codegraph
environment-file: ./jupyter/environment.yml
auto-activate-base: false
use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!

- name: Conda environment info
shell: bash -el {0}
run: conda info

- name: Setup temp directory if missing
run: mkdir -p ./temp

- name: Setup Cache for "temp/downloads" folder
uses: actions/cache@v4
with:
path: ./temp/downloads
key:
${{ runner.os }}-${{ hashFiles('**/*.sh') }}

- name: Download ${{ env.PROJECT_NAME }}-${{ env.REACT_ROUTER_VERSION }}
working-directory: temp
run: |
mkdir -p ${{ env.PROJECT_NAME }}-${{ env.REACT_ROUTER_VERSION }}
cd ${{ env.PROJECT_NAME }}-${{ env.REACT_ROUTER_VERSION }}
echo "Working directory: $( pwd -P )"
./../../scripts/downloader/downloadReactRouter.sh ${{ env.REACT_ROUTER_VERSION }}

- name: Analyze ${{ env.PROJECT_NAME }}-${{ env.REACT_ROUTER_VERSION }}
working-directory: temp/${{ env.PROJECT_NAME }}-${{ env.REACT_ROUTER_VERSION }}
# Shell type can be skipped if jupyter notebook reports (and therefore conda) aren't needed
shell: bash -el {0}
env:
NEO4J_INITIAL_PASSWORD: ${{ secrets.NEO4J_INITIAL_PASSWORD }}
ENABLE_JUPYTER_NOTEBOOK_PDF_GENERATION: "true"
run: |
./../../scripts/analysis/analyze.sh

- name: Move reports from the temp to the results directory preserving their surrounding directory
working-directory: temp
run: ./../scripts/copyReportsIntoResults.sh

# Upload logs and unfinished reports in case of an error for troubleshooting
- name: Archive failed run with logs and unfinished results
if: failure()
uses: actions/upload-artifact@v4
with:
name: code-analysis-logs-java-${{ matrix.java }}-python-${{ matrix.python }}-mambaforge-${{ matrix.mambaforge }}
path: |
./temp/**/runtime/*
./temp/**/reports/*
retention-days: 5

# Upload successful results in case they are needed for troubleshooting
- name: Archive successful results
if: success()
uses: actions/upload-artifact@v4
with:
name: code-report-results-java-${{ matrix.java }}-python-${{ matrix.python }}-mambaforge-${{ matrix.mambaforge }}
path: ./results
if-no-files-found: error
retention-days: 5

# Upload Database Export
# Only possible after an export with "./../../scripts/analysis/analyze.sh --report DatabaseCsvExport"
# Won't be done here because of performance and security concerns
#- name: Archive exported database
# uses: actions/upload-artifact@v3
# with:
# name: code-report-database-export-${{ matrix.java }}-python-${{ matrix.python }}-mambaforge-${{ matrix.mambaforge }}
# path: ./temp/**/import
# if-no-files-found: error
# retention-days: 5

# Commit and push the native image agent results
- name: Display environment variable "github.event_name"
run: echo "github.event_name=${{ github.event_name }}"
- name: Commit "results" directory containing the reports
# Only run when a pull request gets merged or a commit is pushed to the main branch
# git add parameters need to match paths-ignore parameters above
# Git pull before add/commit/push to reduce race conditions on parallel builds
if: github.event_name == 'push'
run: |
git config --global user.name '${{ env.CI_COMMIT_AUTHOR }}'
git config --global user.email '[email protected]'
git config --local http.postBuffer 524288000
git pull
git add results
git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
git push
8 changes: 4 additions & 4 deletions COMMANDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ To run all analysis steps simple execute the following command:
```

👉 See [scripts/examples/analyzeAxonFramework.sh](./scripts/examples/analyzeAxonFramework.sh) as an example script that combines all the above steps.
👉 See [Code Structure Analysis Pipeline](./.github/workflows/code-structure-analysis.yml) on how to do this within a GitHub Actions Workflow.
👉 See [Code Structure Analysis Pipeline](./.github/workflows/java-code-analysis.yml) on how to do this within a GitHub Actions Workflow.

### Command Line Options

Expand Down Expand Up @@ -87,7 +87,7 @@ Change into the [scripts](./scripts/) directory e.g. with `cd scripts` and then
Change into the [results](./results/) directory e.g. with `cd results` and then execute the script [generateCsvReportReference.sh](./scripts/documentation/generateCsvReportReference.sh) with the following command:

👉**Note:** This script is automatically triggered at the end of [copyReportsIntoResults.sh](./scripts/copyReportsIntoResults.sh)
which is included in the pipeline [code-structure-analysis.yml](.github/workflows/code-structure-analysis.yml) and doesn't need to be executed manually normally.
which is included in the pipeline [java-code-analysis.yml](.github/workflows/java-code-analysis.yml) and doesn't need to be executed manually normally.

```script
./../scripts/documentation/generateCsvReportReference.sh
Expand All @@ -98,7 +98,7 @@ which is included in the pipeline [code-structure-analysis.yml](.github/workflow
Change into the [results](./results/) directory e.g. with `cd results` and then execute the script [generateJupyterReportReference.sh](./scripts/documentation/generateJupyterReportReference.sh) with the following command:

👉**Note:** This script is automatically triggered at the end of [copyReportsIntoResults.sh](./scripts/copyReportsIntoResults.sh)
which is included in the pipeline [code-structure-analysis.yml](.github/workflows/code-structure-analysis.yml) and doesn't need to be executed manually normally.
which is included in the pipeline [java-code-analysis.yml](.github/workflows/java-code-analysis.yml) and doesn't need to be executed manually normally.

```script
./../scripts/documentation/generateJupyterReportReference.sh
Expand All @@ -109,7 +109,7 @@ which is included in the pipeline [code-structure-analysis.yml](.github/workflow
Change into the [results](./results/) directory e.g. with `cd results` and then execute the script [generateImageReference.sh](./scripts/documentation/generateImageReference.sh) with the following command:

👉**Note:** This script is automatically triggered at the end of [copyReportsIntoResults.sh](./scripts/copyReportsIntoResults.sh)
which is included in the pipeline [code-structure-analysis.yml](.github/workflows/code-structure-analysis.yml) and doesn't need to be executed manually normally.
which is included in the pipeline [java-code-analysis.yml](.github/workflows/java-code-analysis.yml) and doesn't need to be executed manually normally.

```script
./../scripts/documentation/generateImageReference.sh
Expand Down
2 changes: 1 addition & 1 deletion GETTING_STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,4 +81,4 @@ Please read through the [Prerequisites](./README.md#🛠-prerequisites) in the [
Then open your browser and login to your [local Neo4j Web UI](http://localhost:7474/browser) with "neo4j" as user and the initial password you've chosen.

👉 See [scripts/examples/analyzeAxonFramework.sh](./scripts/examples/analyzeAxonFramework.sh) as an example script that combines all the above steps.
👉 See [Code Structure Analysis Pipeline](./.github/workflows/code-structure-analysis.yml) on how to do this within a GitHub Actions Workflow.
👉 See [Code Structure Analysis Pipeline](./.github/workflows/java-code-analysis.yml) on how to do this within a GitHub Actions Workflow.
Loading