Rework Workflow

[valentinedwv]

Thinking that we might consider reworking the workflow.
Right now we do a few corrections, like the context when we put the Harvested JSONLD documents in summoned s3 path.

Thinking that we might put the uncorrected documents in summoned path, then have a processed/enhanced s3 path, where the documents are corrected, and perhaps formatted into a single style (compact or flattened). Then have a standardized path where we have further processed the documents so they can be loaded.

• Original JSONLD > summoned
• summoned> Context and other processing > processed -- this would be a valid document that can be directly converted to RDF reliably.
• processed > standardization processing > standardized
• standardized > transform > other metadata formats

Standardizing:

• convert to a standard form for dataloading (using frames... which were problematic when bad contexts occur)
• split datacatalogs into a datacatalog and datasets
• extract organization/people/etc from JSONLD, store in separate paths in repo.
• enhance by adding keywords, etc to records.

Standardize is for our systems, it might even be accomplished with a set of s3 events, and lambda serverless functions.. which would be a 'tool' in gleaner.

@fils @nein09 any thoughts?