initial general schema added

[drom]

No description provided.

[indutny]

Not sure why jscs did not complain, but the indent is off here.

[drom]

Sure. Will fix.

[indutny]

Few nits, otherwise looking good. Thank you!

[indutny]

Looking good!

[indutny]

I think we don't need this for now. Just removed it from design.md

[drom]

How about a method on pipeline object like p.validate() that would do pipeline validation.

[indutny]

@drom what exactly should it validate?

[drom]

for the start it should call tv4.validate() to validate current state of the p object for the schema compliance. But we can extend it with formal graph connectivity checks.

[indutny]

@drom p has more high-level representation, it can't be validated against JSON Schema.

[indutny]

@drom I agree about graph correctness, though. There is no place where irreducible loops are supported at the moment. It would be great to check this.

[drom]

@indutny about p object. I understand that we have to call p.render('json') under the hood to get the object that we can validate against the schema.

[indutny]

Yeah, exactly :) I think it might be a good idea to validate it in tests, but exposing function sounds unreasonable.

[drom]

@indutny for the graph topology part. Out of the list below; let me know what properties would you like to check, and what bullets are correct by construction:

1. all links are referencing an object.
   1. nodes[].inputs
   2. nodes[].control
   3. cfg.blocks[].node
   4. cfg.blocks[].nodes[]
   5. cfg.blocks[].successors[]
2. graph is 'connected' (the is no unreachable vertices)
   1. nodes
   2. cfg
3. detect irreducible loops in CFG
4. detect all loops in nodes[].input

[indutny]

I think all links are surely referencing the blocks and nodes. This is guaranteed by JSONFormat parser.

Other points make sense! Thanks!

[drom]

@indutny can we assume that all nodes are topologically sorted?

[indutny]

@drom nope :)

[indutny]

We have .reindex() for this.

[drom]

@indutny both loops and irreducible loops can be detected during toposort. Do you perform it anywhere in the flow?

[indutny]

It is an optional step, might be performed if some of the stages needs it (like linearscan).