Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore]: Make TSConfig stricter and fix as many errors as possible #1596

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

inlined
Copy link
Member

@inlined inlined commented Jan 12, 2025

As part of #1591 I've added a bit more stringency to Genkit's tsconfigs. This is required to add a genkit dependency to firebase-functions as it currently compiles cleanly with skipLibCheck: false explicitly in customers' default templates; asking customers to change this would be a breaking change.

Common fixes included removing unused or non-existent imports/exports, removing dead code, typing the results of response.json(), adding missing libraries, and in one case adding a missing return statement. I started by modifying the base tsconfig.json and then reenabling (only) skipLibCheck on packages that are broken due to downstream dependencies, with a comment describing the reason why the setting is necessary. As a follow-up, we can file bugs and/or make contributions to those downstream libraries to further lessen our dependency on skipLibCheck.

This change ensures skipLibCheck is not necessary for the following packages:

  • genkit
  • @genkit-ai/ai
  • @genkit-ai/core
  • @genkit-ai/checks
  • @genkit-ai/dev-local-vectorstore
  • @genkit-ai/dotprompt
  • @genkit-ai/evaluators
  • @genkit-ai/express
  • @genkit-ai/googleai
  • @genkit-ai/ollama
  • @genkit-ai/firebase
  • @genkit-ai/google-cloud

This change does not fix the following pacakges:

  • @genkit-ai/chroma
  • @genkit-ai/langchain
  • @genkit-ai/pinecone
  • @genkit-ai/vertexai
  • @genkit-ai/mcp

Issues with the remaining packages:

  • firebase, google-cloud, langchain are all blocked on the fact that "long" (a dependency of protobufjs) doesn't do import/require correctly EDIT firebase and google-cloud were fixed with an update to the long library, but langchain has a smattering of errors listed below.
  • chromadb triest to import packages that are not shipped with it (I suspect they need to move their types package from a devDep to a straight dep)
  • pinecone tries to import types that don't exist
  • MCP seems to be blocked on us. We have one error that assertNotificationCompatibility is non-conforming because our method type is never. We have four errors because ts-up is turning our type imports into normal imports and dropping the with { 'resolution-mode': 'import' } annotation, which breaks the CJS .d.ts. I've confirmed that editing the output lib file fixes the issue but it doesn't seem ts-up can be made to preserve imports correctly.

Checklist (if applicable):

Tested by verifying the following program compiles as both a commonJS and ESM module:

import { Document, genkit, z, ModelArgument } from "genkit";
import { BaseEvalDataPoint, Score } from "genkit/evaluator";
import { googleAI } from "@genkit-ai/googleai";
import { ollama } from "genkitx-ollama";
import { checksMiddleware, ChecksEvaluationMetricType } from "@genkit-ai/checks";
import { devLocalIndexerRef } from "@genkit-ai/dev-local-vectorstore";

const ai = genkit({
  promptDir: ".",
  plugins: [
   googleAI(),
   ollama({
      models: [
         {
            name: "gemma",
            type: "generate",
         }
      ],
      serverAddress: "http://127.0.0.1:11434",
   }),
  ],
});

const indexer = devLocalIndexerRef("Dadabase");

(async () => {
   await ai.index({
      indexer,
      documents: [
         Document.fromText("What is Beethoven's favorite fruit? Bananana!"),
         Document.fromText("What do lawyers wear to court? Lawsuits!"),
         Document.fromText("How do astronauts pay for coffee? Star bucks!"),
      ],
   });
});

export const jokeTeller = ai.defineFlow({
   name: "jokeTeller",
   inputSchema: z.string().optional(),
   outputSchema: z.string(),
}, async (type, cb) => {
   const { stream, response } = await ai.generateStream({
     prompt: `Tell me a ${type ?? "dad"} joke.`,
     use: [
      checksMiddleware({
         authOptions: { projectId: "project-id" },
         metrics: [
            ChecksEvaluationMetricType.SEXUALLY_EXPLICIT,
            ChecksEvaluationMetricType.HATE_SPEECH,
         ],
      }),
     ],
   });
   for await (const chunk of stream) {
      cb(chunk);
   }
   return (await response).text;
})

const isFunny = ai.prompt("isFunny");

export async function funnyScore<CustomModelOptions extends z.ZodTypeAny>(
   judgeLlm: ModelArgument<CustomModelOptions>,
   dataPoint: BaseEvalDataPoint,
   judgeConfig?: CustomModelOptions
): Promise<Score> {
   const response = await ai.generate(isFunny.render({output: dataPoint.output!}));
   return {
      score: response.output,
      details: { reasoning: "because this is fake" },
   };
}

@inlined inlined requested review from pavelgj and mbleigh January 12, 2025 04:52
@inlined
Copy link
Member Author

inlined commented Jan 13, 2025

Doing a social network crawl, I've managed to get ahold of someone who could fix the long library. This should fix the firebase, google-cloud, and langchain plugins. Will extend my tests to verify.

@inlined
Copy link
Member Author

inlined commented Jan 13, 2025

Welp; langchain produced a lot of errors still:

node_modules/@langchain/core/dist/language_models/chat_models.d.ts:75:129 - error TS2304: Cannot find name 'this'.

75     _generateCached({ messages, cache, llmStringKey, parsedOptions, handledOptions, }: ChatModelGenerateCachedParameters<typeof this>): Promise<LLMResult & {
                                                                                                                                   ~~~~

node_modules/@langchain/core/dist/language_models/llms.d.ts:67:129 - error TS2304: Cannot find name 'this'.

67     _generateCached({ prompts, cache, llmStringKey, parsedOptions, handledOptions, runId, }: LLMGenerateCachedParameters<typeof this>): Promise<LLMResult & {
                                                                                                                                   ~~~~

node_modules/@langchain/core/tracers/base.d.cts:1:15 - error TS1479: The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("../dist/tracers/base.js")' call instead.

1 export * from '../dist/tracers/base.js'
                ~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/@langchain/openai/dist/azure/chat_models.d.ts:2:10 - error TS2305: Module '"@langchain/core/language_models/chat_models"' has no exported member 'LangSmithParams'.

2 import { LangSmithParams, type BaseChatModelParams } from "@langchain/core/language_models/chat_models";
           ~~~~~~~~~~~~~~~

node_modules/@langchain/openai/dist/chat_models.d.ts:6:25 - error TS2305: Module '"@langchain/core/language_models/chat_models"' has no exported member 'LangSmithParams'.

6 import { BaseChatModel, LangSmithParams, type BaseChatModelParams } from "@langchain/core/language_models/chat_models";
                          ~~~~~~~~~~~~~~~

node_modules/langchain/evaluation.d.cts:1:15 - error TS1479: The current file is a CommonJS module whose imports will produce 'require' calls; however, the referenced file is an ECMAScript module and cannot be imported with 'require'. Consider writing a dynamic 'import("./dist/evaluation/index.js")' call instead.

1 export * from './dist/evaluation/index.js'
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I got firebase and google-cloud to work, though I had to kick some lockfiles. It might be worth someone else double checking my work. New code added:

import { enableFirebaseTelemetry } from "@genkit-ai/firebase";
import { enableGoogleCloudTelemetry } from "@genkit-ai/google-cloud";

enableFirebaseTelemetry();
enableGoogleCloudTelemetry();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

1 participant