[generator] initial unit test support and refactoring (#282)

Context: https://github.com/grendello/JavaBindingGenerator

@grendello has an initial prototype of a "re-imagined" `generator`. Our
current goal is to refactor the current `generator` and slowly migrate
the existing code to look something like the new one. It obviously is
going to take a while, so we need to do this in a fashion where we can
keep things working while making changes over time. Eventually we may
get to a place where we can import @grendello's existing project to be
used within `generator`.

The high-level changes we should first adopt from the new `generator`:

  - An object model that uses POCOs, such as the
    `Java.Interop.Bindings.Syntax` hierarchy @grendello came up with
  - Logical separation between the `api.xml` parsing and the code
    generation
  - The possibility of "unit" tests, instead of the slow/bulky tests we
    have currently, which are in fact "integration" tests
  - The possibility for new backends that will have breaking changes,
    but allow us to move `generator` forward into the future

This PR is the first step in achieving this, here is a breakdown of
the changes:

  - Moved existing tests to an `Integration-Tests` folder
  - New tests will go in the `Unit-Tests` folder if they are indeed
    unit tests
  - `[assembly: InternalsVisibleTo("generator-Tests")]` will need to
    be added to `generator` so that we can write unit tests against
    `internal` classes and methods
  - Instances of `StreamWriter sw` are changed to `TextWriter writer`
    as needed. This allows us to use a `StringWriter` in unit tests
  - A first set of unit tests are validating the simplest cases such
    as class handles, class invoker handles, and fields
  - A set of "support types" are added to the `generator-Tests` project.
    Once the existing code is reasonably under unit test, these types can
    completely go away as we switch the object model over to POCOs.
  - Any generation methods of `Field` have been moved to the
    `CodeGenerator` base class. This allows us to get `Field` closer
    to a POCO.
  - The constructor of `Parameter` is now `internal` so it can be
    instantiated from unit tests
  - Two sets of new tests are added: `JavaInteropCodeGeneratorTests`
    and `XamarinAndroidCodeGeneratorTests`, unit tests within take
    around 1ms, so we can add as many of these as we like
  - Usage of `CodeGenerator.ContextType` are being refactored during
    the migration, as it seems to be a code smell. While writing each
    unit test, I could easily discover the `ContextTypes` stack usage
    because exceptions are thrown. Hopefully, we can remove
    `ContextType` completely as refactoring continues, and just pass
    the required value through an additional parameter.

NOTE: in this PR, I've (hopefully) not altered the output of
`generator`. I have noticed a few bugs/oddities in the output, but plan
to fix these in future PRs after we have these tests.

Miscellaneous changes:

  - VS for Windows unit test runner wants to add a `<Service />` MSBuild
    item, I'm not sure what this is for, but it is a hassle to keep
    discarding it
  - Updated `.gitignore` for VS for Windows

Author: jonathanpeppers

.gitignore

@@ -9,4 +9,5 @@ xa-gendarme.html
 packages
 .vs/
 *.userprefs
+*.user
 Resource.designer.cs

tools/generator/CodeGenerator.cs

@@ -695,22 +695,57 @@ protected CodeGenerator ()
 		{
 		}
 
-		internal    abstract    void    WriteClassHandle (ClassGen type,    StreamWriter sw,    string indent,  CodeGenerationOptions opt,  bool    requireNew);
+		internal    abstract    void    WriteClassHandle (ClassGen type,    TextWriter writer,    string indent,  CodeGenerationOptions opt,  bool    requireNew);
 
-		internal    abstract    void    WriteClassHandle (InterfaceGen type,            StreamWriter sw,    string indent,  CodeGenerationOptions opt,  string  declaringType);
+		internal    abstract    void    WriteClassHandle (InterfaceGen type,            TextWriter writer,    string indent,  CodeGenerationOptions opt,  string  declaringType);
 
-		internal    abstract    void    WriteClassInvokerHandle (ClassGen type,         StreamWriter sw,    string indent,  CodeGenerationOptions opt,  string  declaringType);
-		internal    abstract    void    WriteInterfaceInvokerHandle (InterfaceGen type, StreamWriter sw,    string indent,  CodeGenerationOptions opt,  string  declaringType);
+		internal    abstract    void    WriteClassInvokerHandle (ClassGen type,         TextWriter writer,    string indent,  CodeGenerationOptions opt,  string  declaringType);
+		internal    abstract    void    WriteInterfaceInvokerHandle (InterfaceGen type, TextWriter writer,    string indent,  CodeGenerationOptions opt,  string  declaringType);
 
-		internal    abstract    void    WriteConstructorIdField (Ctor ctor, StreamWriter sw,    string indent,  CodeGenerationOptions opt);
-		internal    abstract    void    WriteConstructorBody (Ctor ctor,    StreamWriter sw,    string indent,  CodeGenerationOptions opt, StringCollection call_cleanup);
+		internal    abstract    void    WriteConstructorIdField (Ctor ctor, TextWriter writer,    string indent,  CodeGenerationOptions opt);
+		internal    abstract    void    WriteConstructorBody (Ctor ctor,    TextWriter writer,    string indent,  CodeGenerationOptions opt, StringCollection call_cleanup);
 
-		internal    abstract    void    WriteMethodIdField (Method method,  StreamWriter sw,    string indent,  CodeGenerationOptions opt);
-		internal    abstract    void    WriteMethodBody (Method method,     StreamWriter sw,    string indent,  CodeGenerationOptions opt);
+		internal    abstract    void    WriteMethodIdField (Method method,  TextWriter writer,    string indent,  CodeGenerationOptions opt);
+		internal    abstract    void    WriteMethodBody (Method method,     TextWriter writer,    string indent,  CodeGenerationOptions opt);
 
-		internal    abstract    void    WriteFieldIdField (Field field,     StreamWriter sw,    string indent,  CodeGenerationOptions opt);
-		internal    abstract    void    WriteFieldGetBody (Field field,     StreamWriter sw,    string indent,  CodeGenerationOptions opt);
-		internal    abstract    void    WriteFieldSetBody (Field field,     StreamWriter sw,    string indent,  CodeGenerationOptions opt);
+		internal    abstract    void    WriteFieldIdField (Field field,     TextWriter writer,    string indent,  CodeGenerationOptions opt);
+		internal    abstract    void    WriteFieldGetBody (Field field,     TextWriter writer,    string indent,  CodeGenerationOptions opt, GenBase type);
+		internal    abstract    void    WriteFieldSetBody (Field field,     TextWriter writer,    string indent,  CodeGenerationOptions opt, GenBase type);
+
+		internal    virtual     void    WriteField (Field field,            TextWriter writer,    string indent,  CodeGenerationOptions opt, GenBase type)
+		{
+			if (field.IsEnumified)
+				writer.WriteLine ("[global::Android.Runtime.GeneratedEnum]");
+			if (field.NeedsProperty) {
+				string fieldType = field.Symbol.IsArray ? "IList<" + field.Symbol.ElementType + ">" : opt.GetOutputName (field.Symbol.FullName);
+				WriteFieldIdField (field, writer, indent, opt);
+				writer.WriteLine ();
+				writer.WriteLine ("{0}// Metadata.xml XPath field reference: path=\"{1}/field[@name='{2}']\"", indent, type.MetadataXPathReference, field.JavaName);
+				writer.WriteLine ("{0}[Register (\"{1}\"{2})]", indent, field.JavaName, field.AdditionalAttributeString ());
+				writer.WriteLine ("{0}{1} {2}{3} {4} {{", indent, field.Visibility, field.IsStatic ? "static " : String.Empty, fieldType, field.Name);
+				writer.WriteLine ("{0}\tget {{", indent);
+				WriteFieldGetBody (field, writer, indent + "\t\t", opt, type);
+				writer.WriteLine ("{0}\t}}", indent);
+
+				if (!field.IsConst) {
+					writer.WriteLine ("{0}\tset {{", indent);
+					WriteFieldSetBody (field, writer, indent + "\t\t", opt, type);
+					writer.WriteLine ("{0}\t}}", indent);
+				}
+				writer.WriteLine ("{0}}}", indent);
+			}
+			else {
+				writer.WriteLine ("{0}// Metadata.xml XPath field reference: path=\"{1}/field[@name='{2}']\"", indent, type.MetadataXPathReference, field.JavaName);
+				writer.WriteLine ("{0}[Register (\"{1}\"{2})]", indent, field.JavaName, field.AdditionalAttributeString ());
+				if (field.IsDeprecated)
+					writer.WriteLine ("{0}[Obsolete (\"{1}\")]", indent, field.DeprecatedComment);
+				if (field.Annotation != null)
+					writer.WriteLine ("{0}{1}", indent, field.Annotation);
+
+				// the Value complication is due to constant enum from negative integer value (C# compiler requires explicit parenthesis).
+				writer.WriteLine ("{0}{1} const {2} {3} = ({2}) {4};", indent, field.Visibility, opt.GetOutputName (field.Symbol.FullName), field.Name, field.Value.Contains ('-') && field.Symbol.FullName.Contains ('.') ? '(' + field.Value + ')' : field.Value);
+			}
+		}
 	}
 }
 

tools/generator/Field.cs

@@ -234,45 +234,6 @@ internal ParameterList SetParameters {
 
 		public string Annotation { get; internal set; }
 
-		void GenerateProperty (StreamWriter sw, string indent, CodeGenerationOptions opt, GenBase gen)
-		{
-			string type = Symbol.IsArray ? "IList<" + Symbol.ElementType + ">" : opt.GetOutputName (Symbol.FullName);
-			opt.CodeGenerator.WriteFieldIdField (this, sw, indent, opt);
-			sw.WriteLine ();
-			sw.WriteLine ("{0}// Metadata.xml XPath field reference: path=\"{1}/field[@name='{2}']\"", indent, gen.MetadataXPathReference, JavaName);
-			sw.WriteLine ("{0}[Register (\"{1}\"{2})]", indent, JavaName, this.AdditionalAttributeString ());
-			sw.WriteLine ("{0}{1} {2}{3} {4} {{", indent, Visibility, IsStatic ? "static " : String.Empty, type, Name);
-			sw.WriteLine ("{0}\tget {{", indent);
-			opt.CodeGenerator.WriteFieldGetBody (this, sw, indent + "\t\t", opt);
-			sw.WriteLine ("{0}\t}}", indent);
-
-			if (!IsConst) {
-				sw.WriteLine ("{0}\tset {{", indent);
-				opt.CodeGenerator.WriteFieldSetBody (this, sw, indent + "\t\t", opt);
-				sw.WriteLine ("{0}\t}}", indent);
-			}
-			sw.WriteLine ("{0}}}", indent);
-		}
-
-		public void Generate (StreamWriter sw, string indent, CodeGenerationOptions opt, GenBase type)
-		{
-			if (IsEnumified)
-				sw.WriteLine ("[global::Android.Runtime.GeneratedEnum]");
-			if (NeedsProperty)
-				GenerateProperty (sw, indent, opt, type);
-			else {
-				sw.WriteLine ("{0}// Metadata.xml XPath field reference: path=\"{1}/field[@name='{2}']\"", indent, type.MetadataXPathReference, JavaName);
-				sw.WriteLine ("{0}[Register (\"{1}\"{2})]", indent, JavaName, this.AdditionalAttributeString ());
-				if (IsDeprecated)
-					sw.WriteLine ("{0}[Obsolete (\"{1}\")]",  indent, DeprecatedComment);
-				if (Annotation != null)
-					sw.WriteLine ("{0}{1}", indent, Annotation);
-
-				// the Value complication is due to constant enum from negative integer value (C# compiler requires explicit parenthesis).
-				sw.WriteLine ("{0}{1} const {2} {3} = ({2}) {4};", indent, Visibility, opt.GetOutputName (Symbol.FullName), Name, Value.Contains ('-') && Symbol.FullName.Contains ('.') ? '(' + Value + ')' : Value);
-			}
-		}
-
 		public bool Validate (CodeGenerationOptions opt, GenericParameterDefinitionList type_params)
 		{
 			symbol = SymbolTable.Lookup (TypeName, type_params);

tools/generator/GenBase.cs

@@ -577,7 +577,7 @@ public bool GenFields (StreamWriter sw, string indent, CodeGenerationOptions opt
 						seen.Add (f.Name);
 					needsProperty = needsProperty || f.NeedsProperty;
 					sw.WriteLine ();
-					f.Generate (sw, indent, opt, this);
+					opt.CodeGenerator.WriteField (f, sw, indent, opt, this);
 				}
 			}
 			return needsProperty;