tech-insights: Unable to create custom fact retriever

[jmpinto-critical]

Workspace

tech-insights

📜 Description

I followed the documentation present here and here and am having some issues creating a custom fact retriever.

👍 Expected behavior

It should show the custom fact I introduced in the Tech Insights view:

👎 Actual Behavior with Screenshots

I get the following error in the backstage logs:

With the corresponding behavior in the Backstage UI:

👟 Reproduction steps

1. Add the backend plugins:

    "@backstage-community/plugin-tech-insights-backend": "^2.4.1",
    "@backstage-community/plugin-tech-insights-backend-module-jsonfc": "^0.6.1",
    "@backstage-community/plugin-tech-insights-node": "^2.5.1",

2. Add the code for the custom fact retriever as stated in the documentation links mentioned previously:

// packages/backend/src/index.ts

import { FactRetriever, techInsightsFactRetrieversExtensionPoint } from '@backstage-community/plugin-tech-insights-node';
import { CatalogClient } from '@backstage/catalog-client';

export const myFactRetriever: FactRetriever = {
  id: 'documentation-number-factretriever', // unique identifier of the fact retriever
  version: '1.0.0', // SemVer version number of this fact retriever schema. This should be incremented if the implementation changes
  entityFilter: [{ kind: 'service' }], // EntityFilter to be used in the future (creating checks, graphs etc.) to figure out which entities this fact retrieves data for.
  schema: {
    // Name/identifier of an individual fact that this retriever returns
    examplenumberfact: {
      type: 'integer', // Type of the fact
      description: 'A fact of a number', // Description of the fact
    },
  },
  handler: async (ctx:any) => {
    // Handler function that retrieves the fact
    const { discovery, config, logger, auth } = ctx;

    const { token } = await auth.getPluginRequestToken({
      onBehalfOf: await auth.getOwnServiceCredentials(),
      targetPluginId: 'catalog',
    });

    const catalogClient = new CatalogClient({
      discoveryApi: discovery,
    });
    const entities = await catalogClient.getEntities(
      {
        filter: [{ kind: 'component' }],
      },
      { token },
    );
    /**
     * snip: Do complex logic to retrieve facts from external system or calculate fact values
     */

    // Respond with an array of entity/fact values
    return entities.items.map((it:any) => {
      return {
        // Entity information that this fact relates to
        entity: {
          namespace: it.metadata.namespace,
          kind: it.kind,
          name: it.metadata.name,
        },

        // All facts that this retriever returns
        facts: {
          examplenumberfact: 2,
        },
        // (optional) timestamp to use as a Luxon DateTime object
      };
    });
  },
};

export const techInsightsModuleMyFactRetriever = createBackendModule({
  pluginId: 'tech-insights',
  moduleId: 'my-fact-retriever',
  register(reg:any) {
    reg.registerInit({
      deps: {
        providers: techInsightsFactRetrieversExtensionPoint,
      },
      async init({ providers }:any) {
        providers.addFactRetrievers({
          myFactRetriever,
        });
      },
    });
  },
});

backend.add(import('@backstage-community/plugin-tech-insights-backend'));
backend.add(import('@backstage-community/plugin-tech-insights-backend-module-jsonfc'));
backend.add(techInsightsModuleMyFactRetriever);

Add the necessary configuration to app-config.yaml as stated in the documentation mentioned previously:

techInsights:
  factRetrievers:
    entityMetadataFactRetriever:
      cadence: '*/15 * * * *'  # Every 15 minutes
      lifecycle: 
        timeToLive: { weeks: 2 }
    entityOwnershipFactRetriever:
      cadence: '0 2 * * *'     # Daily at 2 AM
      lifecycle: 
        maxItems: 10
    techdocsFactRetriever:
      cadence: '0 */6 * * *'   # Every 6 hours
      lifecycle:
        timeToLive: { days: 30 }
    documentation-number-factretriever:
      cadence: '*/15 * * * *'   # Every 15 minutes
      lifecycle:
        timeToLive: { days: 2 }
  factChecker:
    checks:
      groupOwnerCheck:
        type: json-rules-engine
        name: 'Group Owner Check'
        description: 'Verifies that a component has a group owner'
        factIds:
          - entityOwnershipFactRetriever
        rule:
          conditions:
            all:
              - fact: hasGroupOwner
                operator: equal
                value: true
      titleCheck:
        type: json-rules-engine
        name: 'Entity Title Check'
        description: 'Verifies that entities have descriptive titles'
        factIds:
          - entityMetadataFactRetriever
        rule:
          conditions:
            all:
              - fact: hasTitle
                operator: equal
                value: true
              - fact: hasDescription
                operator: equal
                value: true
      myCustomCheck:
        type: json-rules-engine
        name: 'Repository Quality Standards'
        description: 'Ensures repositories meet basic quality standards'
        factIds:
          - documentation-number-factretriever
        rule:
          conditions:
            all:
              - fact: examplenumberfact
                operator: equal
                value: 0

📃 Provide the context for the Bug.

I'm trying to implement a custom fact retriever so that I can define custom scores on my backstage entities.

👀 Have you spent some time to check if this bug has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

Are you willing to submit PR?

None

[awanlin]

@punkle @Xantier can you help with this please? 🙏

[Xantier]

Thanks @jmpinto-critical , this looks like it could be an issue with your configuration. It would be useful if you could contact on Discord with more information.

[jmpinto-critical]

Could you clarify a bit more on that? The configuration I am using is the one in the prints, which I just took from the docs mentioned in the links. Unfortunately, I am unable to use discord at this time, I would be very grateful if we could discuss the issue here.
What do you suspect could be the issue with my configuration?

[jmpinto-critical]

I can attach the full app-config.yaml file if you think it would help.

[jmpinto-critical]

Don't mind any secrets in the file as they all point to local development instances of the tools/integrations.

app-config.yaml

[jmpinto-critical]

@Xantier What additional information would you require to help me out finding the issue with this?

[jmpinto-critical]

I've been trying to debug this in more detail and I have found this:

Even though I register myFactRetriever (the one shown above), and both console.log() calls print to the console, there is no corresponding fact schema created in the database and I get the following log:

[backend]: �[2m2025-11-03T11:58:24.875Z�[22m �[34mtech-insights�[39m �[33mwarn�[39m Validation failed for check Repository Quality Standards. Reference to value examplenumberfact does not exists in referred fact schemas: documentationNumberFactRetriever

Registration:

export const techInsightsModuleMyFactRetriever = createBackendModule({
  pluginId: 'tech-insights',
  moduleId: 'my-fact-retriever',
  register(reg:any) {
    console.log("[register]...");

    reg.registerInit({
      deps: {
        providers: techInsightsFactRetrieversExtensionPoint,
      },
      async init({ providers }:any) {
        console.log("[init]...");
        providers.addFactRetrievers({
          myFactRetriever,
        });
      },
    });
  },
});

Fact Retriever:

export const myFactRetriever: FactRetriever = {
  id: 'documentationNumberFactRetriever', // unique identifier of the fact retriever
  title: 'Dummy Fact Retriever',
  description: 'Generates facts which indicate the completeness of entity metadata',
  version: '0.1.3', // SemVer version number of this fact retriever schema. This should be incremented if the implementation changes
  entityFilter: [{ kind: 'component' }], // EntityFilter to be used in the future (creating checks, graphs etc.) to figure out which entities this fact retrieves data for.
  schema: {
    // Name/identifier of an individual fact that this retriever returns
    examplenumberfact: {
      type: 'integer', // Type of the fact
      description: 'A fact of a number', // Description of the fact
    },
  },
  handler: async ctx => {
    // Handler function that retrieves the fact
    const { discovery, config, logger, auth } = ctx;

    const { token } = await auth.getPluginRequestToken({
      onBehalfOf: await auth.getOwnServiceCredentials(),
      targetPluginId: 'catalog',
    });

    const catalogClient = new CatalogClient({
      discoveryApi: discovery,
    });
    const entities = await catalogClient.getEntities(
      {
        filter: [{ kind: 'component' }],
      },
      { token },
    );
    /**
     * snip: Do complex logic to retrieve facts from external system or calculate fact values
     */

    // Respond with an array of entity/fact values
    return entities.items.map(it => {
      return {
        // Entity information that this fact relates to
        entity: {
          namespace: it.metadata.namespace,
          kind: it.kind,
          name: it.metadata.name,
        },

        // All facts that this retriever returns
        facts: {
          examplenumberfact: 2,
        },
        // (optional) timestamp to use as a Luxon DateTime object
      };
    });
  },
};

[jmpinto-critical]

If the only way to get support is through Discord, I am willing to do it, who can I contact?

[nunolopesit-jpg]

I also have this issue. Could you please try to assist?