Skip to content

Add Microsoft.ML.GenAI.Phi, test package and sample project. #7184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 43 commits into from
Jul 30, 2024
Merged

Add Microsoft.ML.GenAI.Phi, test package and sample project. #7184

merged 43 commits into from
Jul 30, 2024

Conversation

@LittleLittleCloud
Copy link
Member

@LittleLittleCloud LittleLittleCloud commented Jun 28, 2024 •
edited
Loading

We are excited to review your PR.

So we can do the best job, please check:

  • There's a descriptive title that will make sense to other developers some time from now.
  • There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
  • Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
  • You have included any necessary tests in the same PR.

#7169

@LittleLittleCloud LittleLittleCloud mentioned this pull request Jun 28, 2024
Open
13 tasks
LittleLittleCloud
LittleLittleCloud commented Jun 28, 2024
View reviewed changes
LittleLittleCloud
LittleLittleCloud commented Jun 28, 2024
View reviewed changes
tarekgh
tarekgh reviewed Jun 28, 2024
View reviewed changes
@build-analysis build-analysis bot mentioned this pull request Jul 1, 2024
Open
3 tasks
@azure-pipelines
Copy link

azure-pipelines bot commented Jul 19, 2024

Azure Pipelines successfully started running 2 pipeline(s).

tarekgh
tarekgh reviewed Jul 22, 2024
View reviewed changes
tarekgh
tarekgh reviewed Jul 22, 2024
View reviewed changes
@luisquintanilla
Copy link
Contributor

luisquintanilla commented Jul 22, 2024

@LittleLittleCloud samples look good.

Something we might want to think about is the naming of the SK naming conventions for adding a model to make it more generic.

Similar to:

https://github.com/feiyun0112/SemanticKernel.Connectors.OnnxRuntimeGenAI/blob/6efcbcfbb84c5ecdb5cc31215903b6df13f3f841/OnnxRuntimeGenAIServiceCollectionExtensions.cs#L26

tarekgh
tarekgh reviewed Jul 22, 2024
View reviewed changes
src/Microsoft.ML.GenAI.Core/Pipeline/CausalLMPipeline.cs

return tokens
.Where(t => t.Offset != (0, 0))
.Select(t => t.Id)
Copy link
Member

@tarekgh tarekgh Jul 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what this for?

Copy link
Member Author

@LittleLittleCloud LittleLittleCloud Jul 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To detect the _ token automatically added by tokenzier

Copy link
Member

@tarekgh tarekgh Jul 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, thanks. it is worth a comment.

Also, EncodeToToken is very expensive comparing to EncodeToIds. If you are interested to optimize I can suggest some code to check for _ token.

Copy link
Member Author

@LittleLittleCloud LittleLittleCloud Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that would be very helpful

@LittleLittleCloud
Copy link
Member Author

LittleLittleCloud commented Jul 23, 2024

@LittleLittleCloud samples look good.

Something we might want to think about is the naming of the SK naming conventions for adding a model to make it more generic.

Similar to:

https://github.com/feiyun0112/SemanticKernel.Connectors.OnnxRuntimeGenAI/blob/6efcbcfbb84c5ecdb5cc31215903b6df13f3f841/OnnxRuntimeGenAIServiceCollectionExtensions.cs#L26

Rename to AddGenAIChatCompletion and AddGenAITextCompletion

@build-analysis build-analysis bot mentioned this pull request Jul 23, 2024
Open
@build-analysis build-analysis bot mentioned this pull request Jul 29, 2024
Open
@LittleLittleCloud
Copy link
Member Author

LittleLittleCloud commented Jul 30, 2024

/azp run

@azure-pipelines
Copy link

azure-pipelines bot commented Jul 30, 2024

Azure Pipelines successfully started running 2 pipeline(s).

@codecov
Copy link

codecov bot commented Jul 30, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 50.28902% with 774 lines in your changes missing coverage. Please review.

Project coverage is 68.90%. Comparing base (579fe03) to head (a97afdf).
Report is 10 commits behind head on main.

Files Patch % Lines
...crosoft.ML.GenAI.Core/Pipeline/CausalLMPipeline.cs 0.00% 98 Missing ⚠️
src/Microsoft.ML.GenAI.Phi/Module/Phi3Attention.cs 25.00% 86 Missing and 1 partial ⚠️
src/Microsoft.ML.GenAI.Phi/Utils.cs 12.94% 72 Missing and 2 partials ⚠️
src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs 19.04% 68 Missing ⚠️
src/Microsoft.ML.GenAI.Phi/Module/Phi2Model.cs 12.16% 65 Missing ⚠️
.../Microsoft.ML.GenAI.Phi/Module/Phi3DecoderLayer.cs 16.66% 59 Missing and 1 partial ⚠️
src/Microsoft.ML.GenAI.Phi/Module/Phi2Attention.cs 32.09% 53 Missing and 2 partials ⚠️
...crosoft.ML.GenAI.Phi/Module/Phi3RotaryEmbedding.cs 20.00% 32 Missing ⚠️
...ML.GenAI.Phi/Module/Phi3SuScaledRotaryEmbedding.cs 24.39% 31 Missing ⚠️
...c/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMAgent.cs 56.71% 21 Missing and 8 partials ⚠️
... and 18 more
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #7184      +/-   ##
==========================================
- Coverage   69.06%   68.90%   -0.16%     
==========================================
  Files        1401     1441      +40     
  Lines      268516   270828    +2312     
  Branches    27812    28019     +207     
==========================================
+ Hits       185451   186619    +1168     
- Misses      75902    77004    +1102     
- Partials     7163     7205      +42
Flag Coverage Δ
Debug 68.90% <50.28%> (-0.16%) ⬇️
production 63.43% <41.62%> (-0.22%) ⬇️
test 89.02% <95.60%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...crosoft.ML.GenAI.Core/Extension/ModuleExtension.cs 54.22% <100.00%> (ø)
src/Microsoft.ML.GenAI.Core/Module/GenAILinear.cs 100.00% <100.00%> (ø)
....ML.GenAI.Phi/Extension/SemanticKernelExtension.cs 100.00% <100.00%> (ø)
...Microsoft.ML.GenAI.Phi/Phi2/Phi2TokenizerHelper.cs 100.00% <100.00%> (ø)
...Microsoft.ML.GenAI.Phi/Phi3/Phi3TokenizerHelper.cs 100.00% <100.00%> (ø)
test/Microsoft.ML.GenAI.Phi.Tests/AutoGenTests.cs 100.00% <100.00%> (ø)
...icrosoft.ML.GenAI.Phi.Tests/SemanticKernelTests.cs 100.00% <100.00%> (ø)
src/Microsoft.ML.GenAI.Phi/Phi2/Config.cs 98.00% <98.00%> (ø)
...rosoft.ML.GenAI.Core.Tests/QuantizedLinearTests.cs 95.52% <95.52%> (ø)
src/Microsoft.ML.GenAI.Phi/Phi3/Config.cs 93.84% <93.84%> (ø)
... and 25 more

... and 38 files with indirect coverage changes

@LittleLittleCloud LittleLittleCloud merged commit 6d1f7e2 into dotnet:main Jul 30, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Aug 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@tarekgh tarekgh tarekgh left review comments

@ericstj ericstj Awaiting requested review from ericstj

+1 more reviewer

@michaelgsharp michaelgsharp michaelgsharp approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@LittleLittleCloud LittleLittleCloud

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@LittleLittleCloud @luisquintanilla @tarekgh @michaelgsharp