diff --git a/COMMANDS.md b/COMMANDS.md index 5c39bdd71..62938d59d 100644 --- a/COMMANDS.md +++ b/COMMANDS.md @@ -22,7 +22,7 @@ - [Setup Neo4j Graph Database](#setup-neo4j-graph-database) - [Start Neo4j Graph Database](#start-neo4j-graph-database) - [Setup jQAssistant Java Code Analyzer](#setup-jqassistant-java-code-analyzer) - - [Download Maven Artifacts to Analyze](#download-maven-artifacts-to-analyze) + - [Download Maven Artifacts to analyze](#download-maven-artifacts-to-analyze) - [Reset the database and scan the java artifacts](#reset-the-database-and-scan-the-java-artifacts) - [Database Queries](#database-queries) - [Cypher Shell](#cypher-shell) @@ -52,7 +52,8 @@ To run all analysis steps simple execute the following command: ./../../scripts/analysis/analyze.sh ``` -👉 See [scripts/examples/analyzeAxonFramework.sh](./scripts/examples/analyzeAxonFramework.sh) as an example script that combines all the above steps. +👉 See [scripts/examples/analyzeAxonFramework.sh](./scripts/examples/analyzeAxonFramework.sh) as an example script that combines all the above steps for a Java Project. +👉 See [scripts/examples/analyzeReactRouter.sh](./scripts/examples/analyzeReactRouter.sh) as an example script that combines all the above steps for a Typescript Project. 👉 See [Code Structure Analysis Pipeline](./.github/workflows/java-code-analysis.yml) on how to do this within a GitHub Actions Workflow. ### Command Line Options diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index fe6aa8ff5..b363f7bb4 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -80,5 +80,6 @@ Please read through the [Prerequisites](./README.md#hammer_and_wrench-prerequisi Then open your browser and login to your [local Neo4j Web UI](http://localhost:7474/browser) with "neo4j" as user and the initial password you've chosen. -👉 See [scripts/examples/analyzeAxonFramework.sh](./scripts/examples/analyzeAxonFramework.sh) as an example script that combines all the above steps. +👉 See [scripts/examples/analyzeAxonFramework.sh](./scripts/examples/analyzeAxonFramework.sh) as an example script that combines all the above steps for a Java Project. +👉 See [scripts/examples/analyzeReactRouter.sh](./scripts/examples/analyzeReactRouter.sh) as an example script that combines all the above steps for a Typescript Project. 👉 See [Code Structure Analysis Pipeline](./.github/workflows/java-code-analysis.yml) on how to do this within a GitHub Actions Workflow. \ No newline at end of file diff --git a/README.md b/README.md index 6674c0b23..0b51db815 100644 --- a/README.md +++ b/README.md @@ -26,13 +26,13 @@ Contained within this repository is a comprehensive and automated code graph ana Here is an overview of reports made with [Jupyter Notebooks](https://jupyter.org). For a detailed reference see [Jupyter Notebook Report Reference](#page_with_curl-jupyter-notebook-report-reference -- [External Dependencies](./results/AxonFramework-4.9.3/external-dependencies/ExternalDependencies.md) contains detailed information about external library usage ([Notebook](./jupyter/ExternalDependenciesJava.ipynb)). -- [Internal Dependencies](./results/AxonFramework-4.9.3/internal-dependencies/InternalDependencies.md) is based on [Analyze java package metrics in a graph database](https://joht.github.io/johtizen/data/2023/04/21/java-package-metrics-analysis.html) and also includes cyclic dependencies ([Notebook](./jupyter/InternalDependenciesJava.ipynb)). -- [Method Metrics](./results/AxonFramework-4.9.3/method-metrics/MethodMetrics.ipynb) shows how the effective number of lines of code and the cyclomatic complexity are distributed across the methods in the code ([Notebook](./jupyter/MethodMetricsJava.ipynb)). -- [Node Embeddings](./results/AxonFramework-4.9.3/node-embeddings/NodeEmbeddings.md) shows how to generate node embeddings and to further reduce their dimensionality to be able to visualize them in a 2D plot ([Notebook](./jupyter/NodeEmbeddingsJava.ipynb)). -- [Object Oriented Design Quality Metrics](./results/AxonFramework-4.9.3/object-oriented-design-metrics/ObjectOrientedDesignMetrics.md) is based on [OO Design Quality Metrics by Robert Martin](https://api.semanticscholar.org/CorpusID:18246616) ([Notebook](./jupyter/ObjectOrientedDesignMetricsJava.ipynb)). -- [Overview](./results/AxonFramework-4.9.3/overview/Overview.md) contains overall statistics and details about methods and their complexity. ([Notebook](./jupyter/OverviewJava.ipynb)). -- [Visibility Metrics](./results/AxonFramework-4.9.3/visibility-metrics/VisibilityMetrics.md) ([Notebook](./jupyter/VisibilityMetricsJava.ipynb)). +- [External Dependencies](./results/AxonFramework-4.9.3/external-dependencies-java/ExternalDependenciesJava.md) contains detailed information about external library usage ([Notebook](./jupyter/ExternalDependenciesJava.ipynb)). +- [Internal Dependencies](./results/AxonFramework-4.9.3/internal-dependencies-java/InternalDependenciesJava.md) is based on [Analyze java package metrics in a graph database](https://joht.github.io/johtizen/data/2023/04/21/java-package-metrics-analysis.html) and also includes cyclic dependencies ([Notebook](./jupyter/InternalDependenciesJava.ipynb)). +- [Method Metrics](./results/AxonFramework-4.9.3/method-metrics-java/MethodMetricsJava.md) shows how the effective number of lines of code and the cyclomatic complexity are distributed across the methods in the code ([Notebook](./jupyter/MethodMetricsJava.ipynb)). +- [Node Embeddings](./results/AxonFramework-4.9.3/node-embeddings-java/NodeEmbeddingsJava.md) shows how to generate node embeddings and to further reduce their dimensionality to be able to visualize them in a 2D plot ([Notebook](./jupyter/NodeEmbeddingsJava.ipynb)). +- [Object Oriented Design Quality Metrics](./results/AxonFramework-4.9.3/object-oriented-design-metrics-java/ObjectOrientedDesignMetricsJava.md) is based on [OO Design Quality Metrics by Robert Martin](https://api.semanticscholar.org/CorpusID:18246616) ([Notebook](./jupyter/ObjectOrientedDesignMetricsJava.ipynb)). +- [Overview](./results/AxonFramework-4.9.3/overview-java/OverviewJava.md) contains overall statistics and details about methods and their complexity. ([Notebook](./jupyter/OverviewJava.ipynb)). +- [Visibility Metrics](./results/AxonFramework-4.9.3/visibility-metrics-java/VisibilityMetricsJava.md) ([Notebook](./jupyter/VisibilityMetricsJava.ipynb)). - [Wordcloud](./results/AxonFramework-4.9.3/wordcloud/Wordcloud.md) contains a visual representation of package and class names ([Notebook](./jupyter/Wordcloud.ipynb)). ### :book: Graph Data Science Reports diff --git a/jupyter/NodeEmbeddingsJava.ipynb b/jupyter/NodeEmbeddingsJava.ipynb index b1335be55..d2c4afac7 100644 --- a/jupyter/NodeEmbeddingsJava.ipynb +++ b/jupyter/NodeEmbeddingsJava.ipynb @@ -258,9 +258,16 @@ " # See https://bobbyhadz.com/blog/python-attributeerror-list-object-has-no-attribute-shape\n", " embeddings_as_numpy_array = np.array(embeddings.embedding.to_list())\n", "\n", + " # The parameter \"perplexity\" needs to be smaller than the sample size\n", + " # See https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html\n", + " number_of_nodes=embeddings.shape[0]\n", + " perplexity = min(number_of_nodes - 1.0, 30.0)\n", + " print(\"t-SNE: Sample size (Number of nodes)={size}\".format(size = number_of_nodes))\n", + " print(\"t-SNE: perplexity={perplexity}\".format(perplexity=perplexity))\n", + "\n", " # Use t-distributed stochastic neighbor embedding (t-SNE) to reduce the dimensionality \n", " # of the previously calculated node embeddings to 2 dimensions for visualization\n", - " t_distributed_stochastic_neighbor_embedding = TSNE(n_components=2, verbose=1, random_state=50)\n", + " t_distributed_stochastic_neighbor_embedding = TSNE(n_components=2, perplexity=perplexity, verbose=1, random_state=50)\n", " two_dimension_node_embeddings = t_distributed_stochastic_neighbor_embedding.fit_transform(embeddings_as_numpy_array)\n", " display(two_dimension_node_embeddings.shape) # Display the shape of the t-SNE result\n", "\n", diff --git a/jupyter/NodeEmbeddingsTypescript.ipynb b/jupyter/NodeEmbeddingsTypescript.ipynb index 7524d6661..b65333e0d 100644 --- a/jupyter/NodeEmbeddingsTypescript.ipynb +++ b/jupyter/NodeEmbeddingsTypescript.ipynb @@ -258,9 +258,16 @@ " # See https://bobbyhadz.com/blog/python-attributeerror-list-object-has-no-attribute-shape\n", " embeddings_as_numpy_array = np.array(embeddings.embedding.to_list())\n", "\n", + " # The parameter \"perplexity\" needs to be smaller than the sample size\n", + " # See https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html\n", + " number_of_nodes=embeddings.shape[0]\n", + " perplexity = min(number_of_nodes - 1.0, 30.0)\n", + " print(\"t-SNE: Sample size (Number of nodes)={size}\".format(size = number_of_nodes))\n", + " print(\"t-SNE: perplexity={perplexity}\".format(perplexity=perplexity))\n", + "\n", " # Use t-distributed stochastic neighbor embedding (t-SNE) to reduce the dimensionality \n", " # of the previously calculated node embeddings to 2 dimensions for visualization\n", - " t_distributed_stochastic_neighbor_embedding = TSNE(n_components=2, verbose=1, random_state=50)\n", + " t_distributed_stochastic_neighbor_embedding = TSNE(n_components=2, perplexity=perplexity, verbose=1, random_state=50)\n", " two_dimension_node_embeddings = t_distributed_stochastic_neighbor_embedding.fit_transform(embeddings_as_numpy_array)\n", " display(two_dimension_node_embeddings.shape) # Display the shape of the t-SNE result\n", "\n", diff --git a/scripts/examples/analyzeReactRouter.sh b/scripts/examples/analyzeReactRouter.sh new file mode 100755 index 000000000..190017062 --- /dev/null +++ b/scripts/examples/analyzeReactRouter.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +# This is an example for the analysis of a the Typescript project "react-router". +# It includes the creation of the temporary directory, the working directory, the artifacts download and the analysis itself. + +# Note: The first (and only) parameter is the version of "react-router" to analyze. +# Note: This script is meant to be started in the root directory of this repository. + +# Fail on any error ("-e" = exit on first error, "-o pipefail" exist on errors within piped commands) +set -o errexit -o pipefail + +# Read the first input argument containing the version of the artifacts +if [ "$#" -ne 1 ]; then + echo "analyzerReactRouter Error: Usage: $0 " >&2 + exit 1 +fi +projectVersion=$1 + +# Check if environment variable is set +if [ -z "${NEO4J_INITIAL_PASSWORD}" ]; then + echo "analyzerReactRouter: Error: Requires environment variable NEO4J_INITIAL_PASSWORD to be set first. Use 'export NEO4J_INITIAL_PASSWORD='." + exit 1 +fi + +# Create the temporary directory for all analysis projects. +mkdir -p ./temp +cd ./temp + +# Create the working directory for this specific analysis. +mkdir -p "./react-router-${projectVersion}" +cd "./react-router-${projectVersion}" + +# Create the artifacts directory that will contain the code to be analyzed. +mkdir -p ./artifacts + +# Download AxonFramework artifacts (jar files) from Maven +./../../scripts/downloader/downloadReactRouter.sh "${projectVersion}" + +# Start the analysis +./../../scripts/analysis/analyze.sh \ No newline at end of file