Supporting testing between Nuget dependencies

[Shazwazza]

This is a WIP to resolve the task #290

I've got this working with the CsProjCoreToolchain and have added a sample showing the syntax and how it works. During development i noticed that the BuildScriptFilePath doesn't ever seem to be used so raised a question/issue about that here #804

What I need help understanding at this stage is how to get this working with the RoslynToolchain. I can see that the Roslyn Generator calls into csc and there's a /references switch so I assume I'd need to find a way to go fetch the nuget packages and get these wired up but I'm not sure if there's some sort of automated way like with the dotnet cli that can be used. Any ideas on this one?

Please provide any feedback you might have. Hopefully i haven't done something too odd, there's a bit of a learning curve involved in this codebase :)

[adamsitnik]

@Shazwazza the PR looks very promising!

Could you please add the missing { and answer to my question?

[adamsitnik]

@Shazwazza big thanks for doing this!

[adamsitnik]

@Shazwazza please excuse me for a late reply

On Monday we should release 0.11.0, then I take a week off and after that, I will most probably merge your PR to a separate branch and test it.

[Shazwazza]

@adamsitnik no worries at all, i've been on holidays too :)

[Rbn3D]

@Shazwazza Does this PR allow benchmark between two versions of the same project, but one as project reference (source) and the other as nuget?

This is my exact use case.

I'm trying to test performance optimizations i'm doing for a existing nuget project, and the ideal way to go would be to benchmark the nuget version agaisnt the source version directly.

This way I can add the benchmark project to the repo, and the performance of the changes I'm doing could be measured easily, without even having to commmit, or publish to nuget. Just like running a test suite :)

Any ideas?

Thanks in advance.

[Shazwazza]

@Rbn3D This PR indeed would let you benchmark between different versions of your product but it's not really intended to work with a reference and also Nugets, it works only with Nuget references but this will still work for you since i'm sure you can make a Nuget package of your current version.

The way it would need to work is that your benchmark project would reference your minimum version of your product that has the same API surface area so you can write your benchmarks using those APIs, then you would create each benchmark to run against specific nuget versions which would load in the newer versions of your product and execute the newer versions of those same APIs. See here for more details: #805 (comment)

[Shazwazza]

@adamsitnik Hi adam, just bumping this one :) I realize this is out of date with latest changes so happy to try to get this updated, but seeing as though I'd love a hacktoberfest t-shirt, could i close this PR and submit a new one? ;)

[adamsitnik]

@Shazwazza sure thing!

[AndreyAkinshin]

could i close this PR and submit a new one? ;)

I close this PR.