Skip to content

Commit 3a1c4cb

Browse files
JohTJohT
authored
Merge pull request #223 from JohT/feature/enrich-with-version-numbers
Support Typescript version numbers
2 parents ce471da + c2a9ac8 commit 3a1c4cb

18 files changed

+368
-22
lines changed

‎COMMANDS.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,9 @@ The [analyze.sh](./scripts/analysis/analyze.sh) command comes with these command
7171
- `--report Csv` only generates CSV reports. This speeds up the report generation and doesn't depend on Python, Jupyter Notebook or any other related dependencies. The default value os `All` to generate all reports. `Jupiter` will only generate Jupyter Notebook reports. `DatabaseCsvExport` exports the whole graph database as a CSV file (performance intense, check if there are security concerns first).
7272

7373
- `--profile Neo4jv4` uses the older long term support (june 2023) version v4.4.x of Neo4j and suitable compatible versions of plugins and JQAssistant. `Neo4jv5` will explicitly select the newest (june 2023) version 5.x of Neo4j. Without setting
74-
a profile, the newest versions will be used. Profiles are scripts that can be found in the directory [scripts/profiles](./scripts/profiles/).
74+
a profile, the newest versions will be used. Profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
75+
76+
- `--profile Neo4jv5-continue-on-scan-errors` is based on the default profile (`Neo4jv5`) but uses the jQAssistant configuration template [template-neo4jv5-jqassistant-continue-on-error.yaml](./scripts/configuration/template-neo4jv5-jqassistant-continue-on-error.yaml) to continue on scan error instead of failing fast. This is temporarily useful when there is a known error that needs to be ignored. It is still recommended to use the default profile and fail fast if there is something wrong. Profiles can be found in the directory [scripts/profiles](./scripts/profiles/).
7577

7678
- `--explore` activates the "explore" mode where no reports are generated. Furthermore, Neo4j won't be stopped at the end of the script and will therefore continue running. This makes it easy to just set everything up but then use the running Neo4j server to explore the data manually.
7779

‎README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,14 @@ The [Code Structure Analysis Pipeline](./.github/workflows/java-code-analysis.ym
226226
TYPESCRIPT_SCAN_HEAP_MEMORY=16384 ./../../scripts/analysis/analyze.sh
227227
```
228228

229+
- How can i continue on errors when scanning Typescript projects instead of cancelling the whole analysis?
230+
👉 Use the profile `Neo4jv5-continue-on-scan-errors` (default = `Neo4jv5`):
231+
232+
```shell
233+
./../../scripts/analysis/analyze.sh --profile Neo4jv5-continue-on-scan-errors
234+
```
235+
236+
229237
## 🕸 Web References
230238

231239
- [Graph Data Science 101: Understanding Graphs and Graph Data Science](https://techfirst.medium.com/graph-data-science-101-understanding-graphs-and-graph-data-science-c25055a9db01)

‎cypher/Artifact_Dependencies/Set_maven_artifact_version.cypher‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
// Set property "version" on Artifact nodes to what is specified in the Maven POM
2+
3+
MATCH (artifact:Java:Artifact)-[:CONTAINS]-(pom:Maven:Pom)
4+
MATCH (pom)-[:HAS_ROOT_ELEMENT]->(pomProject:Element{name:'project'})
5+
OPTIONAL MATCH (pomProject)-[:HAS_ELEMENT]->(:Element{name:'version'})-[:HAS_TEXT]->(versionValue:Text)
6+
OPTIONAL MATCH (pomProject)-[:HAS_ELEMENT]->(:Element{name:'parent'})
7+
-[:HAS_ELEMENT]->(:Element{name:'version'})-[:HAS_TEXT]->(parentVersionValue:Text)
8+
WITH *, artifact, coalesce(versionValue.value, parentVersionValue.value) AS versionNumber
9+
WHERE versionNumber IS NOT NULL
10+
SET artifact.version = versionNumber
11+
RETURN count(DISTINCT artifact.fileName) AS numberOfUpdatedArtifacts
12+
// Debugging
13+
// RETURN artifact.name, versionNumber, versionValue.value, parentVersionValue.value
14+
// LIMIT 25

‎cypher/Internal_Dependencies/Inter_scan_and_project_dependencies_of_Typescript_modules.cypher‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
// Inter scan/project higher-level module dependencies for manual exploration
2+
3+
MATCH (source_module:TS:Module)-[:DEPENDS_ON]->(target_module:TS:Module)
4+
MATCH (source_project:TS:Project)-[:CONTAINS]->(source_module)
5+
MATCH (target_project:TS:Project)-[:CONTAINS]->(target_module)
6+
OPTIONAL MATCH (source_project)<-[:REFERENCED_PROJECTS*1..3]-(source_project_roots:TS:Project)
7+
MATCH (source_scan:TS:Scan)-[:CONTAINS_PROJECT]->(source_project)
8+
MATCH (target_scan:TS:Scan)-[:CONTAINS_PROJECT]->(target_project)
9+
RETURN source_scan, target_scan
10+
,collect(source_project)[0..2], collect(target_project)[0..2]
11+
,collect(source_module)[0..9], collect(target_module)[0..9]
12+
LIMIT 50

‎cypher/Typescript_Enrichment/Add_RESOLVES_TO_relationship_for_matching_modules.cypher‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,13 @@ MATCH (module:TS:Module)
77
WHERE module.globalFqn IS NOT NULL
88
MATCH (externalModule:TS:ExternalModule)
99
WHERE module.globalFqn IS NOT NULL
10-
AND (module.globalFqn = externalModule.globalFqn
11-
OR ( externalModule.name = module.name
10+
AND ((module.globalFqn = externalModule.globalFqn)
11+
OR (module.module = externalModule.module)
12+
OR ( externalModule.name = module.name
1213
AND externalModule.moduleName = module.moduleName
1314
AND externalModule.namespace = module.namespace
1415
AND externalModule.extensionExtended = module.extensionExtended
15-
)
16+
)
1617
)
1718
AND module <> externalModule
1819
CALL { WITH module, externalModule

‎cypher/Typescript_Enrichment/Add_module_properties.cypher‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ OPTIONAL MATCH (class:TS:Class)-[:DECLARES]->(ts)
1818
,coalesce('@' + nullif(namespaceName, ''), '') AS namespaceNameWithAtPrefixed
1919
,replace(symbolName, coalesce(optionalClassName + '.', ''), '') AS symbolNameWithoutClassName
2020
SET ts.namespace = namespaceNameWithAtPrefixed
21-
,ts.module = modulePathName
21+
,ts.module = modulePathNameWithoutIndexAndDefault
2222
,ts.moduleName = moduleName
2323
,ts.name = coalesce(symbolNameWithoutClassName, indexAndExtensionOmittedName)
2424
,ts.extensionExtended = moduleNameExtensionExtended

‎cypher/Typescript_Enrichment/Count_internal_modules_resolving_external_ones.cypher‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
// How many internal modules resolve/represent external ones (for manual exploration)?
2+
3+
MATCH (internal_module:TS:Module)
4+
WITH count(internal_module) AS totalNumberOfInternalModules
5+
,collect(internal_module) AS internal_modules
6+
UNWIND internal_modules AS internal_module
7+
MATCH (external_module:TS:ExternalModule)-[:RESOLVES_TO]->(internal_module)
8+
WHERE NOT external_module.isNodeModule = true
9+
RETURN totalNumberOfInternalModules
10+
,COUNT { (all_external_modules:TS:ExternalModule)
11+
WHERE NOT all_external_modules.isNodeModule = true}
12+
AS totalNumberOfExternalModules
13+
,count(DISTINCT external_module) AS numberOfResolvedExternalModules
14+
,collect(external_module.globalFqn + ' -> ' + internal_module.globalFqn)[0..4] AS exampleFullNames

‎cypher/Typescript_Enrichment/Link_external_modules_to_corresponding_npm_dependency.cypher‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
// Link ExternalModule nodes to their corresponding NPM Dependency if available
2+
3+
MATCH (external_module:TS:ExternalModule{isNodeModule:true})
4+
WITH count(external_module) AS totalNumberOfExternalModules
5+
,collect(external_module) AS external_modules
6+
UNWIND external_modules AS external_module
7+
MATCH (external_module)<-[:USES]-(project:TS:Project)
8+
MATCH (project)-[:HAS_NPM_PACKAGE]->(npm_package:NPM:Package)-[npm_dependency_relation]->(npm_dependency:NPM:Dependency)
9+
WHERE external_module.globalFqn CONTAINS ('/node_modules/' + npm_dependency.name)
10+
CALL { WITH npm_dependency, external_module
11+
MERGE (external_module)-[:PROVIDED_BY_NPM_DEPENDENCY]->(npm_dependency)
12+
SET external_module.npmPackage = npm_dependency.name
13+
,external_module.npmPackageVersion = npm_dependency.dependency
14+
} IN TRANSACTIONS
15+
RETURN totalNumberOfExternalModules
16+
,count(DISTINCT external_module.globalFqn) AS numberOfLinkedExternalModules
17+
,count(DISTINCT project.name) AS numberOfProjects
18+
//Debugging
19+
//RETURN totalNumberOfExternalModules, project.name, npm_dependency.name, external_module.globalFqn
20+
//LIMIT 40

‎cypher/Typescript_Enrichment/Link_projects_to_npm_packages.cypher‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ MATCH (npmPackage:NPM:Package)
1111
, ''
1212
) AS relativeNpmPackageDirectory
1313
MATCH (project:TS:Project)-[:HAS_CONFIG]->(config:File)<-[:CONTAINS]-(projectConfigDir:Directory)
14-
WHERE projectConfigDir.absoluteFileName ENDS WITH relativeNpmPackageDirectory
14+
WHERE projectConfigDir.absoluteFileName ENDS WITH '/source' + relativeNpmPackageDirectory
1515
WITH npmPackage
1616
,relativeNpmPackageDirectory
1717
,collect(DISTINCT project) AS projects
@@ -25,6 +25,7 @@ MATCH (npmPackage:NPM:Package)
2525
// Set the "relativeFileDirectory" on the npm package to the relative directory
2626
// that contains the package.json file
2727
SET npmPackage.relativeFileDirectory = relativeNpmPackageDirectory
28+
,project.version = npmPackage.version
2829
RETURN count(*) AS numberOfCreatedNpmPackageRelationships
2930
// Detailed results for debugging
3031
//RETURN npmPackage.fileName AS npmPackageFileName

‎scripts/configuration/template-neo4jv5-jqassistant-continue-on-error.yaml‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
jqassistant:
2+
# Controls whether the execution of jQAssistant shall be skipped
3+
#
4+
# -Djqassistant.skip: true|false
5+
skip: false
6+
7+
# Defines the local and remote Maven repositories for retrieving additional plugins.
8+
repositories:
9+
# The path to the local repository.
10+
#
11+
# -Djqassistant.repositories.local (optional)
12+
local: ./../downloads/jqassistant-repository
13+
14+
# The list of plugins to load and activate.
15+
#
16+
# Each plugin is identified using its Maven coordinates:
17+
#
18+
# -Djqassistant.plugins[0].group-id
19+
# -Djqassistant.plugins[0].artifact-id
20+
# -Djqassistant.plugins[0].version
21+
# -Djqassistant.plugins[0].classifier (optional)
22+
# -Djqassistant.plugins[0].type (optional)
23+
#plugins:
24+
# - group-id:
25+
# artifact-id:
26+
# version:
27+
# classifier:
28+
# type:
29+
plugins:
30+
- group-id: org.jqassistant.plugin.typescript
31+
artifact-id: jqassistant-typescript-plugin
32+
version: 1.3.0
33+
- group-id: org.jqassistant.plugin
34+
artifact-id: jqassistant-npm-plugin
35+
version: 2.0.0
36+
- group-id: de.kontext-e.jqassistant.plugin
37+
artifact-id: jqassistant.plugin.git
38+
version: 2.1.0
39+
40+
41+
# The store configuration
42+
store:
43+
# URI of the database to connect to. Supported URI schemes are 'file' for embedded databases and 'bolt' for connecting to a running Neo4j instance (3.x+), e.g.
44+
#
45+
# -Djqassistant.store.uri
46+
#uri: file://target/jqassistant/store
47+
uri: bolt://localhost:${NEO4J_BOLT_PORT:7687}
48+
49+
# Settings for the embedded Neo4j store
50+
embedded:
51+
# Enable the HTTP and BOLT connector for the embedded store
52+
#
53+
# -Djqassistant.store.embedded.connector-enabled: true|false
54+
connector-enabled: false
55+
56+
# The listen address for the HTTP and BOLT connectors
57+
#
58+
# -Djqassistant.store.embedded.listen-address
59+
listen-address: localhost
60+
61+
# The BOLT connector port
62+
#
63+
# -Djqassistant.store.embedded.bolt-port
64+
bolt-port: 7687
65+
66+
# The HTTP connector port
67+
#
68+
# -Djqassistant.store.embedded.http-port: 7474
69+
http-port: 7474
70+
71+
# Settings for connecting to a remote Neo4j store
72+
remote:
73+
# The user name for authentication.
74+
#
75+
# -Djqassistant.store.username
76+
username: ${NEO4J_USER:neo4j}
77+
78+
# The password for authentication.
79+
#
80+
# -Djqassistant.store.password
81+
password: ${NEO4J_INITIAL_PASSWORD}
82+
83+
# Activate encryption level for 'bolt' connections.
84+
#
85+
# -Djqassistant.store.encryption: true|false
86+
encryption: false
87+
88+
#The trust strategy for 'bolt' connections
89+
#
90+
# -Djqassistant.store.trust-strategy: trustAllCertificates|trustCustomCaSignedCertificates|trustSystemCaSignedCertificates
91+
trust-strategy: trustAllCertificates
92+
93+
# The file containing the custom CA certificate for trust strategy.
94+
#
95+
# -Djqassistant.store.trust-certificate
96+
trust-certificate:
97+
98+
# Additional properties to be passed to the remote store as key-value pairs.
99+
#
100+
# -Djqassistant.store.properties
101+
properties:
102+
103+
# The Scanner configuration
104+
scan:
105+
# Indicates whether to initially reset the store (i.e. clear all nodes and relationships) before scanning.
106+
#
107+
# -Djqassistant.scan.reset: true|false
108+
reset: true
109+
110+
# Specifies if a scan shall be continued if an error is encountered.
111+
#
112+
# -Djqassistant.scan.continue-on-error: true|false
113+
continue-on-error: true
114+
115+
# The items to include for scanning.
116+
include:
117+
# A list of files to include.
118+
#
119+
#jqassistant.scan.include.files[0]
120+
files:
121+
# - src/folder
122+
123+
# A list of URLs to include.
124+
#
125+
#jqassistant.scan.include.urls[0]
126+
urls:
127+
# - maven:repository::https://nexus/repository
128+
129+
# The properties to configure scanner plugins as key-value pairs. The supported properties are plugin specific.
130+
#
131+
# -Djqassistant.scan.properties
132+
properties:
133+
# plugin.property.key: value
134+
file.exclude: "*/.DS_Store"
135+
json.file.exclude: "*/.reports/jqa/ts-output.json"
136+
137+
# The analyze configuration
138+
analyze:
139+
# The rule configuration
140+
rule:
141+
# The name of the directory containing project rules.
142+
#
143+
# -Djqassistant.analyze.rule.directory
144+
directory: jqassistant/rules
145+
146+
# The default severity of concepts without an explicit severity.
147+
#
148+
# -Djqassistant.analyze.rule.default-concept-severity: INFO|MINOR|MAJOR|CRITICAL|BLOCKER
149+
default-concept-severity: MINOR
150+
151+
# The default severity of constraint without an explicit severity.
152+
#
153+
# -Djqassistant.analyze.rule.default-constraint-severity: INFO|MINOR|MAJOR|CRITICAL|BLOCKER
154+
default-constraint-severity: MAJOR
155+
156+
# The default severity of groups without an explicit severity.
157+
#
158+
# -Djqassistant.analyze.rule.default-group-severity: INFO|MINOR|MAJOR|CRITICAL|BLOCKER
159+
default-group-severity:
160+
161+
# The report configuration
162+
report:
163+
# The properties to configure report plugins. The supported properties are plugin specific.
164+
#
165+
# -Djqassistant.analyze.report.properties
166+
properties:
167+
# plugin.property.key: value
168+
169+
# Determines the severity level for reporting a warning.
170+
#
171+
# -Djqassistant.analyze.report.warn-on-severity: INFO|MINOR|MAJOR|CRITICAL|BLOCKER|NEVER
172+
warn-on-severity: MINOR
173+
174+
# Determines the severity level for reporting a failure.
175+
#
176+
# -Djqassistant.analyze.report.fail-on-severity: INFO|MINOR|MAJOR|CRITICAL|BLOCKER|NEVER
177+
fail-on-severity: MAJOR
178+
179+
# Determines if jQAssistant shall continue the build if failures have been detected.
180+
#
181+
# -Djqassistant.analyze.report.continue-on-failure: true|false
182+
continue-on-failure: false
183+
184+
# Create an archive containing all generated reports.
185+
#
186+
# -Djqassistant.analyze.report.create-archive: true|false
187+
create-archive: false
188+
189+
# Output directory for analysis reports
190+
directory: ./runtime/jqassistant/report
191+
192+
# The concepts to be applied.
193+
#
194+
# -Djqassistant.analyze.concepts[0]
195+
# https://github.com/jQAssistant/jqa-java-plugin/blob/e17fc3032fb0d4258d4f5ca0c64099a6c789c070/src/main/asciidoc/release-notes.adoc#L9
196+
concepts:
197+
- java-classpath:Resolve
198+
- java:PackageDependency
199+
- java:ArtifactDependency
200+
- java:JavaVersion
201+
202+
# The constraints to be validated.
203+
#
204+
# -Djqassistant.analyze.constraints[0]
205+
constraints:
206+
# - my-constraint
207+
208+
# The groups to be executed.
209+
#
210+
# -Djqassistant.analyze.groups[0]
211+
groups:
212+
# - spring-boot:Default
213+
214+
# The parameters to be passed to the executed rules.
215+
#
216+
# -Djqassistant.analyze.rule-parameters."parameterName"
217+
rule-parameters:
218+
# parameterName: value
219+
220+
# Execute concepts even if they have already been applied before
221+
#
222+
# -Djqassistant.analyze.execute-applied-concepts: true|false
223+
execute-applied-concepts: false
224+
225+
# The execution time [seconds] for rules (concepts/constraints) to show a warning. Can be used as a hint for optimization.
226+
#
227+
# -Djqassistant.analyze.warn-on-rule-execution-time-seconds
228+
warn-on-rule-execution-time-seconds: 5

0 commit comments

Comments
 (0)