[Serialization] Drop a class if the superclass can't be found #24819

jrose-apple · 2019-05-16T01:05:38Z

...instead of crashing. Not a perfect answer because it opens us up to cyclic dependencies that weren't there before (see the new validation-test cases), but probably a net win in practice.

rdar://problem/50125674

jrose-apple · 2019-05-16T01:05:46Z

@swift-ci Please test

jrose-apple · 2019-05-16T01:05:51Z

@swift-ci Please test source compatibility

swift-ci · 2019-05-16T01:19:56Z

Build failed
Swift Test Linux Platform
Git Sha - bb5cbc09ea545cfb921ee57b8f3cc135ecda1256

jrose-apple · 2019-05-16T01:31:57Z

lib/Serialization/Serialization.cpp

Ah, this is a behavior change because now it doesn't check if the extension is in problemContext.

I can fix it but I need to extract the test case from SwiftPM first.

swift-ci · 2019-05-16T01:56:27Z

Build failed
Swift Test OS X Platform
Git Sha - bb5cbc09ea545cfb921ee57b8f3cc135ecda1256

jrose-apple · 2019-05-17T00:57:06Z

@swift-ci Please test

jrose-apple · 2019-05-17T00:57:14Z

@swift-ci Please test source compatibility

jrose-apple · 2019-05-17T00:57:45Z

@swift-ci Please test compiler performance

jrose-apple · 2019-05-17T01:02:37Z

…I didn't actually manage to push the changes. Oops.

swift-ci · 2019-05-17T01:03:17Z

Compilation-performance test failed

jrose-apple · 2019-05-17T01:04:56Z

@swift-ci Please test

jrose-apple · 2019-05-17T01:07:50Z

@swift-ci Please test

swift-ci · 2019-05-17T01:08:19Z

Build failed
Swift Test OS X Platform
Git Sha - bb5cbc09ea545cfb921ee57b8f3cc135ecda1256

jrose-apple · 2019-05-17T01:09:28Z

@swift-ci Please test source compatibility

jrose-apple · 2019-05-17T01:09:37Z

@swift-ci Please test compiler performance

swift-ci · 2019-05-17T01:09:53Z

Build failed
Swift Test Linux Platform
Git Sha - bb5cbc09ea545cfb921ee57b8f3cc135ecda1256

swift-ci · 2019-05-17T01:10:13Z

Compilation-performance test failed

...instead of crashing. Also drop the class if its generic requirements depend on a type that can't be loaded (instead of crashing). rdar://problem/50125674

That is, if a struct's generic requirements can't be deserialized, drop the struct. This is the same logic that's already in play for enums and (as of the previous commit) classes, so it should be pretty well tested by now. (Hence the sole test I'm adding here, snuck into superclass.swift because it's a superclass /constraint/ being tested.) I don't know of any outstanding issues caused by this, but it was weird to have it for enums and classes but not structs, so here we are.

jrose-apple · 2019-05-17T17:25:58Z

Summary for master full

Unexpected test results, excluded stats for ProcedureKit, Tagged, Wordy, Deferred

No regressions above thresholds

Debug-batch
- debug-batch brief
- debug-batch detailed
Release
- release brief
- release detailed

Debug-batch

debug-batch brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (0)

name	old	new	delta	delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (3)

name	old	new	delta	delta_pct
Frontend.NumInstructionsExecuted	31,166,988,624,306	30,973,331,285,385	-193,657,338,921	-0.62%
LLVM.NumLLVMBytesOutput	1,168,086,630	1,168,094,636	8,006	0.0%
time.swift-driver.wall	2858.9s	2851.6s	-7.3s	-0.26%

debug-batch detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (2)

name	old	new	delta	delta_pct
Sema.OverriddenDeclsRequest	8,238,831	8,119,393	-119,438	-1.45% ✅
Sema.USRGenerationRequest	13,298,425	13,020,647	-277,778	-2.09% ✅

Unchanged (delta < 1.0% or delta < 100.0ms) (107)

name	old	new	delta	delta_pct
AST.NumASTBytesAllocated	72,838,308,366	72,506,672,477	-331,635,889	-0.46%
AST.NumDecls	94,750	94,750	0	0.0%
AST.NumDependencies	213,731	213,725	-6	-0.0%
AST.NumImportedExternalDefinitions	1,212,037	1,212,037	0	0.0%
AST.NumInfixOperators	35,705	35,705	0	0.0%
AST.NumLinkLibraries	0	0	0	0.0%
AST.NumLoadedModules	255,366	255,366	0	0.0%
AST.NumLocalTypeDecls	123	123	0	0.0%
AST.NumObjCMethods	15,429	15,429	0	0.0%
AST.NumPostfixOperators	18	18	0	0.0%
AST.NumPrecedenceGroups	17,488	17,488	0	0.0%
AST.NumPrefixOperators	90	90	0	0.0%
AST.NumReferencedDynamicNames	122	122	0	0.0%
AST.NumReferencedMemberNames	4,149,543	4,149,543	0	0.0%
AST.NumReferencedTopLevelNames	323,183	323,183	0	0.0%
AST.NumSourceBuffers	406,343	406,343	0	0.0%
AST.NumSourceLines	3,165,267	3,165,267	0	0.0%
AST.NumSourceLinesPerSecond	2,343,308	2,345,641	2,333	0.1%
AST.NumStoredPropertiesQueries	9,367,427	9,367,187	-240	-0.0%
AST.NumTotalClangImportedEntities	4,644,819	4,623,320	-21,499	-0.46%
Driver.ChildrenMaxRSS	111,980,908,544	112,101,189,632	120,281,088	0.11%
Driver.DriverDepCascadingDynamic	0	0	0	0.0%
Driver.DriverDepCascadingExternal	0	0	0	0.0%
Driver.DriverDepCascadingMember	0	0	0	0.0%
Driver.DriverDepCascadingNominal	0	0	0	0.0%
Driver.DriverDepCascadingTopLevel	0	0	0	0.0%
Driver.DriverDepDynamic	0	0	0	0.0%
Driver.DriverDepExternal	0	0	0	0.0%
Driver.DriverDepMember	0	0	0	0.0%
Driver.DriverDepNominal	0	0	0	0.0%
Driver.DriverDepTopLevel	0	0	0	0.0%
Driver.NumDriverJobsRun	19,330	19,330	0	0.0%
Driver.NumDriverJobsSkipped	0	0	0	0.0%
Driver.NumDriverPipePolls	184,993	184,051	-942	-0.51%
Driver.NumDriverPipeReads	201,458	202,044	586	0.29%
Driver.NumProcessFailures	0	0	0	0.0%
Frontend.MaxMallocUsage	744,674,042,704	741,464,592,200	-3,209,450,504	-0.43%
Frontend.NumInstructionsExecuted	31,166,988,624,306	30,973,331,285,385	-193,657,338,921	-0.62%
Frontend.NumProcessFailures	0	0	0	0.0%
IRModule.NumIRAliases	121,583	121,583	0	0.0%
IRModule.NumIRBasicBlocks	4,681,557	4,681,557	0	0.0%
IRModule.NumIRComdatSymbols	0	0	0	0.0%
IRModule.NumIRFunctions	2,132,797	2,132,797	0	0.0%
IRModule.NumIRGlobals	2,238,403	2,238,403	0	0.0%
IRModule.NumIRIFuncs	0	0	0	0.0%
IRModule.NumIRInsts	61,540,049	61,540,049	0	0.0%
IRModule.NumIRNamedMetaData	93,930	93,930	0	0.0%
IRModule.NumIRValueSymbols	3,901,667	3,901,667	0	0.0%
LLVM.NumLLVMBytesOutput	1,168,086,630	1,168,094,636	8,006	0.0%
Parse.NumFunctionsParsed	174,293	174,293	0	0.0%
Parse.NumIterableDeclContextParsed	1,188,598	1,188,598	0	0.0%
SILModule.NumSILGenDefaultWitnessTables	0	0	0	0.0%
SILModule.NumSILGenFunctions	1,100,052	1,100,052	0	0.0%
SILModule.NumSILGenGlobalVariables	41,604	41,604	0	0.0%
SILModule.NumSILGenVtables	12,415	12,415	0	0.0%
SILModule.NumSILGenWitnessTables	46,421	46,421	0	0.0%
SILModule.NumSILOptDefaultWitnessTables	0	0	0	0.0%
SILModule.NumSILOptFunctions	1,565,179	1,565,179	0	0.0%
SILModule.NumSILOptGlobalVariables	42,499	42,499	0	0.0%
SILModule.NumSILOptVtables	20,589	20,589	0	0.0%
SILModule.NumSILOptWitnessTables	101,981	101,981	0	0.0%
Sema.AccessLevelRequest	2,847,454	2,845,492	-1,962	-0.07%
Sema.AttachedPropertyDelegateRequest	1,436,445	1,436,445	0	0.0%
Sema.AttachedPropertyDelegateTypeRequest	279,155	279,155	0	0.0%
Sema.CustomAttrNominalRequest	0	0	0	0.0%
Sema.DefaultAndMaxAccessLevelRequest	63,691	63,683	-8	-0.01%
Sema.DefaultDefinitionTypeRequest	5,928	5,928	0	0.0%
Sema.DefaultTypeRequest	345,097	345,097	0	0.0%
Sema.EnumRawTypeRequest	18,246	18,246	0	0.0%
Sema.ExtendedNominalRequest	4,110,104	4,098,336	-11,768	-0.29%
Sema.InheritedDeclsReferencedRequest	4,627,155	4,584,273	-42,882	-0.93%
Sema.InheritedTypeRequest	241,800	241,437	-363	-0.15%
Sema.IsDynamicRequest	2,215,297	2,215,297	0	0.0%
Sema.IsFinalRequest	4,038,357	4,012,445	-25,912	-0.64%
Sema.IsObjCRequest	1,944,000	1,943,790	-210	-0.01%
Sema.MangleLocalTypeDeclRequest	246	246	0	0.0%
Sema.NamedLazyMemberLoadFailureCount	23,618	23,490	-128	-0.54%
Sema.NamedLazyMemberLoadSuccessCount	21,164,316	21,158,010	-6,306	-0.03%
Sema.NominalTypeLookupDirectCount	35,625,026	35,558,818	-66,208	-0.19%
Sema.NumConformancesDeserialized	6,859,238	6,799,646	-59,592	-0.87%
Sema.NumConstraintScopes	19,639,034	19,631,735	-7,299	-0.04%
Sema.NumConstraintsConsideredForEdgeContraction	52,025,648	52,022,858	-2,790	-0.01%
Sema.NumDeclsDeserialized	52,329,020	51,880,592	-448,428	-0.86%
Sema.NumDeclsFinalized	2,007,584	2,007,584	0	0.0%
Sema.NumDeclsTypechecked	1,033,718	1,033,718	0	0.0%
Sema.NumDeclsValidated	2,395,139	2,395,136	-3	-0.0%
Sema.NumFunctionsTypechecked	951,470	951,470	0	0.0%
Sema.NumGenericSignatureBuilders	1,179,462	1,174,657	-4,805	-0.41%
Sema.NumLazyGenericEnvironments	10,479,885	10,454,160	-25,725	-0.25%
Sema.NumLazyGenericEnvironmentsLoaded	226,113	225,832	-281	-0.12%
Sema.NumLazyIterableDeclContexts	7,263,171	7,244,875	-18,296	-0.25%
Sema.NumLeafScopes	12,972,992	12,967,190	-5,802	-0.04%
Sema.NumTypesDeserialized	17,113,667	17,062,010	-51,657	-0.3%
Sema.NumTypesValidated	1,826,198	1,826,192	-6	-0.0%
Sema.NumUnloadedLazyIterableDeclContexts	4,781,757	4,782,235	478	0.01%
Sema.PropertyDelegateBackingPropertyInfoRequest	276,638	276,638	0	0.0%
Sema.PropertyDelegateBackingPropertyTypeRequest	279,155	279,155	0	0.0%
Sema.PropertyDelegateTypeInfoRequest	0	0	0	0.0%
Sema.RequirementRequest	68,749	68,748	-1	-0.0%
Sema.RequirementSignatureRequest	71,582	71,196	-386	-0.54%
Sema.SelfBoundsFromWhereClauseRequest	6,660,366	6,604,216	-56,150	-0.84%
Sema.SetterAccessLevelRequest	159,215	159,215	0	0.0%
Sema.StructuralTypeRequest	0	0	0	0.0%
Sema.SuperclassDeclRequest	88,872	88,489	-383	-0.43%
Sema.SuperclassTypeRequest	38,525	38,525	0	0.0%
Sema.TypeDeclsFromWhereClauseRequest	33,158	33,150	-8	-0.02%
Sema.UnderlyingTypeDeclsReferencedRequest	177,931	177,035	-896	-0.5%

Release

release brief

Regressed (0)

name	old	new	delta	delta_pct

Improved (0)

name	old	new	delta	delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (3)

name	old	new	delta	delta_pct
Frontend.NumInstructionsExecuted	31,036,488,871,906	31,035,161,557,703	-1,327,314,203	-0.0%
LLVM.NumLLVMBytesOutput	994,184,506	994,184,630	124	0.0%
time.swift-driver.wall	5385.5s	5388.9s	3.4s	0.06%

release detailed

Regressed (0)

name	old	new	delta	delta_pct

Improved (0)

name	old	new	delta	delta_pct

Unchanged (delta < 1.0% or delta < 100.0ms) (22)

name	old	new	delta	delta_pct
AST.NumImportedExternalDefinitions	233,224	233,224	0	0.0%
AST.NumLoadedModules	16,875	16,875	0	0.0%
AST.NumTotalClangImportedEntities	803,155	803,155	0	0.0%
IRModule.NumIRBasicBlocks	3,971,735	3,971,735	0	0.0%
IRModule.NumIRFunctions	1,790,633	1,790,633	0	0.0%
IRModule.NumIRGlobals	1,971,030	1,971,030	0	0.0%
IRModule.NumIRInsts	36,342,810	36,342,810	0	0.0%
IRModule.NumIRValueSymbols	3,493,208	3,493,208	0	0.0%
LLVM.NumLLVMBytesOutput	994,184,506	994,184,630	124	0.0%
SILModule.NumSILGenFunctions	762,562	762,562	0	0.0%
SILModule.NumSILOptFunctions	1,019,891	1,019,891	0	0.0%
Sema.NumConformancesDeserialized	2,354,716	2,354,716	0	0.0%
Sema.NumConstraintScopes	17,773,306	17,773,306	0	0.0%
Sema.NumDeclsDeserialized	6,336,134	6,336,140	6	0.0%
Sema.NumDeclsValidated	1,212,685	1,212,685	0	0.0%
Sema.NumFunctionsTypechecked	365,205	365,205	0	0.0%
Sema.NumGenericSignatureBuilders	200,879	200,879	0	0.0%
Sema.NumLazyGenericEnvironments	1,300,471	1,300,471	0	0.0%
Sema.NumLazyGenericEnvironmentsLoaded	22,496	22,496	0	0.0%
Sema.NumLazyIterableDeclContexts	808,673	808,673	0	0.0%
Sema.NumTypesDeserialized	3,316,066	3,318,031	1,965	0.06%
Sema.NumTypesValidated	751,070	751,070	0	0.0%

jrose-apple · 2019-05-17T17:27:13Z

@swift-ci Please test Linux

jrose-apple · 2019-05-17T17:27:23Z

Everything else already passed, so

@swift-ci Please smoke test macOS

swift-ci · 2019-05-17T17:29:57Z

Build failed
Swift Test Linux Platform
Git Sha - aba4040d2e65fe1b5a994d049b6f92380122cbe5

slavapestov · 2019-05-20T17:01:30Z

lib/Serialization/Deserialization.cpp

+
+    Identifier name = MF.getIdentifier(nameID);
+
+    for (TypeID dependencyID :


Is it right for inheritance clause entries to add dependencies? The change I'm working on just drops ones that don't deserialize. I think only the superclass and raw type of an enum should introduce dependencies.

The inheritance clause doesn't add dependencies; it's just the silly LLVM bitstream container format that forces us to put both dependencies and inherited types in the same array. That's what the slice is for.

…super-no-one-will-be [Serialization] Drop a class if the superclass can't be found (cherry picked from commit 7963529)

…one-will-be (#24915) [Serialization] Drop a class if the superclass can't be found (cherry picked from commit 7963529) rdar://problem/50125674

jrose-apple requested review from harlanhaskins, jckarter and slavapestov May 16, 2019 01:05

jrose-apple commented May 16, 2019

View reviewed changes

jrose-apple force-pushed the when-everyone-is-super-no-one-will-be branch from bb5cbc0 to aba4040 Compare May 17, 2019 01:04

jrose-apple added 2 commits May 17, 2019 10:25

[Serialization] Drop a class if the superclass can't be found

ff7c6f6

...instead of crashing. Also drop the class if its generic requirements depend on a type that can't be loaded (instead of crashing). rdar://problem/50125674

jrose-apple force-pushed the when-everyone-is-super-no-one-will-be branch from aba4040 to 96e8d56 Compare May 17, 2019 17:27

slavapestov approved these changes May 20, 2019

View reviewed changes

slavapestov reviewed May 20, 2019

View reviewed changes

jrose-apple merged commit 7963529 into swiftlang:master May 20, 2019

jrose-apple deleted the when-everyone-is-super-no-one-will-be branch May 20, 2019 18:30

jrose-apple mentioned this pull request May 20, 2019

[5.1] [Serialization] Drop a class if the superclass can't be found #24915

Merged


		Identifier name = MF.getIdentifier(nameID);

		for (TypeID dependencyID :

[Serialization] Drop a class if the superclass can't be found #24819

[Serialization] Drop a class if the superclass can't be found #24819

Uh oh!

Conversation

jrose-apple commented May 16, 2019

Uh oh!

jrose-apple commented May 16, 2019

Uh oh!

jrose-apple commented May 16, 2019

Uh oh!

swift-ci commented May 16, 2019

Uh oh!

jrose-apple May 16, 2019

Choose a reason for hiding this comment

Uh oh!

jrose-apple May 16, 2019

Choose a reason for hiding this comment

Uh oh!

swift-ci commented May 16, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

swift-ci commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

swift-ci commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

swift-ci commented May 17, 2019

Uh oh!

swift-ci commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Summary for master full

Debug-batch

debug-batch brief

debug-batch detailed

Release

release brief

release detailed

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

jrose-apple commented May 17, 2019

Uh oh!

swift-ci commented May 17, 2019

Uh oh!

slavapestov May 20, 2019

Choose a reason for hiding this comment

Uh oh!

jrose-apple May 20, 2019

Choose a reason for hiding this comment

Uh oh!

slavapestov May 20, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants