Skip to content
Canonical

S528 census — owner feedback: stale concepts and context rot

Much of what I’ve found was driven from reviewing code and files in relation to what we had planned for taxonomy_snapshot.json, entity_aliases_snapshot.json, and the related files from the folder these reside in, which led me to review the DBs, reference and runbooks, the ontology CVs, and large amounts of the codebase.

Additional comments/questions

Section titled “Additional comments/questions”

  • Starting with taxonomy_snapshot.json:

    • The file shows 7 taxonomy domains and 34 taxonomy subtopics.
    • There are also 7 content_types, 6 platforms, 7 requirement_type, and 8 form_types.
    • The most important thing to ground this work in is where the concept of the taxonomy_domains and taxonomy_subtopics come from, along with coverage_targets - all three are concepts from the original IMS repo that was forked when the platform was first known as knowledge-hub - this is clear from the DB schema, where both taxonomy-* tables are ONLY connected to coverage_targets, and we’ve already determined that the content-related coverage target functionality can be retired - the only coverage concept that should remain is for identifying how “covered” a form instance is, based on the available q&a pairs for completing it - where any gaps remain, there are then different methods for filling these (manual entry by user, search of wider corpus including OKF, amongst others).
    • If I’m reading the codebase correctly, one of the main purposes for the taxonomy values relates to classification, which again, was carried over from the IMS, and then built upon - for example, classification-architecture.md was last verified in April, and is still very content_items/pre-cocoindex pivot/bid-era focused. Same with entity-type-taxonomy-spec.md.
      • Historically we had ts & python classification - but what are we now “classifying” and to what end, when viewed against the reframe corpus-reframe-review.html?
      • This isn’t to say we shouldn’t have concepts of taxonomy or classification, but rather, I believe these have been misused and proliferated across the codebase in their current form. For example, taxonomy should of course be a layer of our ontology - see phase-b-prerequisite-1-onthology-pipeline.md & phase-b-prerequisite-1-onthology-pipeline-feedback-investigation.md for what was intended when we introduced the ontology to the canonical platform.
    • If you review 01-vision.md, lane-a-workflow-inventory-strawman.md, and corpus-reframe-review.html (especially the admission reframe plus the next “phase” - “Post-v1 — the refinery grows inlets (roadmap themes 15/16)”), I can’t currently see why our platform still needs to be so heavily anchored to what was a very specific IMS use case early on, where IMS’ sole purpose is reviewing content that I’d ingested personally from different platforms (linkedin, reddit, websites) and then would search for useful information, compared to Canonical, which with the introduction of cocoindex and OKF, handles ETL using a different approach.
    • Whilst these are considerably outdated, the following files will provide context on the original client requirement, before the platform evolved into what it is today:
      • kh-client-content-archive/docs/client-briefs/Product_KB_Dev_Brief.md
      • kh-client-content-archive/docs/client-briefs/Sector-Intelligence-Brief-Liam-Final.md
      • kh-client-content-archive/docs/client-briefs/sector-intelligence-analysis.md

  • Building on what I’ve outlined above, this should then provide a grounded view when reviewing the following files, and whether they are required in their current form, or at all. This isn’t a full list, but just an example of where I believe we have duplication or over complexity, or completely unnecessary functionality - the likelihood here is that over time, during the development of the platform, different requirements for ‘seeding’ data have existed, but to my mind, there are three main concepts: (i) data that’s seeded to either a Platform or client staging DB, for example, when a DB reset is required (ii) seeding the initial data that a new client database would require for a new client deployment, and (iii) seeding test data.

    • I don’t believe however, that we would seed entity_aliases, taxonomy data, or canonicalisations (canonicalisation_parity) to a client, as these would be client-specific.
    • There may be some inaccuracies, but client-app-deploy.md is the main runbook for when we do start working with new clients.
    • Subset of files that we need to review and categorise: scripts/propogate-canonical-content.ts scripts/propogation/payload-contract.ts tenant_config file seed-tenant-from-bundle.ts sync-plugin-taxonomy.ts - “content” focus/classification staging-reference-refresh.sh seed-platform-from-staging.ts seed-platform-workspaces.ts seed-platform-feed.ts seed-e2e-users.ts seed-procurement-test-data.ts
      • the generate-* family of scripts
      • lost of related, but unnecessary test files.
    • The knowledge-hub plugin hasn’t been reviewed/updated since pre-rename to canonical, and wouldn’t be a justification for retaining e.g., the taxonomy concepts in their current form, as again, things have changed considerably since it was initially built, and it’s no longer the case that it would be used to classify a content item, for example.

  • Probably the biggest disconnect here is flow between the Ontology CVs -> the DB -> OKF.

    • Firstly, the ontology was originally intended to be synced to the DB, and then used as a linter for OKF.
    • However, having reviewed the OKF ontology CVs, I can see considerable context rot and rather than these being structured canonical records, there has been considerable accretion and also considerable outdated content. I could likely very quickly go through the CVs and strip away elements which I believe to be incorrect or outdated, but there may be a better approach.

  • Fixing these underlying issues would not only strip away considerable wasted effort each time we complete a task, as we’d have a clearer view of what the platform is actually supposed to do, and it would also resolve much of the retire-rewire work that’s in-flight, ensuring we continue with a more grounded north star for decision-making.

    • I can immediately see, for example, that with the ID128 nightly cocoindex integration work that we’ve been going through, there are many skipped tests which certainly aren’t required in their current form. And when we come to review those integration tests to repoint them, which is one of our next tasks, we would avoid having to fix issues like the following:
      • __tests__/integration/cocoindex/url-landing-set.integration.test.ts - “lands the source_documents half of the evidence pair”, “lands the reference_items half of the evidence pair”, and “backlinks every ledger row to the ri id with content_item_id NULL” - there isn’t a half/half approach required here, a reference item is a first-class entity, the same way a source document is.
      • __tests__/integration/cocoindex/op-id-stamping.integration.test.ts (2 tests) 208141ms - “content_items.op_id matches q_a_extractions.op_id for rows from the same run 401ms” - there isn’t a content_items concept anymore.
      • And many others, which could be quickly resolved ocne we have a grounded view to work within.