Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Studio/feature/multiple tests #1020

Merged
merged 29 commits into from
Jul 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions apps/frontend/app/(website)/studio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,28 @@
</a>
</p>

Codemod Studio is an AI workbench for creating codemods. Codemods are powerful code automation bots that can automate many crucial yet tedious coding tasks, such as migrations, upgrades, large-scale changes, enforcing best practices, and bringing conformity to a codebase. However, building codemods manually is a time-consuming process.
Codemod Studio is an AI workbench for creating codemods. Codemods are powerful code automation bots that can automate
many crucial yet tedious coding tasks, such as migrations, upgrades, large-scale changes, enforcing best practices, and
bringing conformity to a codebase. However, building codemods manually is a time-consuming process.

Codemod Studio allows you to “instantly” create code automation bots with the help of AI, specialized helpers, and debuggers, as well as a vibrant community of “codemod champions.”
Codemod Studio allows you to “instantly” create code automation bots with the help of AI, specialized helpers, and
debuggers, as well as a vibrant community of “codemod champions.”

## Features

### 1. Build

With the help of fine-tuned LLMs under-the-hood and codemod creation features such as expert-curated prompts, and smart highlighting, Codemod Studio can help you build codemods in a few minutes.
With the help of fine-tuned LLMs under-the-hood and codemod creation features such as expert-curated prompts, and smart
highlighting, Codemod Studio can help you build codemods in a few minutes.

### 2. Test

Codemod Studio allows you to iteratively test, debug, and improve your codemods as you build them.

### 3. Run

With close integration with Codemod platform, Codemod Studio allows you to easily run your codemods over your projects using Codemod CLI, VS Code Extension, or on your GitHub repositories right from the studio.
With close integration with Codemod platform, Codemod Studio allows you to easily run your codemods over your projects
using Codemod CLI, VS Code Extension, or on your GitHub repositories right from the studio.

## Documentation

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { cn } from "@/utils";
import type { Lightbulb } from "@phosphor-icons/react";
import Tooltip from "@studio/components/Tooltip/Tooltip";
import * as React from "react";

export const ExplainIcon = ({
text,
Icon,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import { useModal } from "../../../src/hooks/useModal";
import { useCodemodExecution } from "../hooks/useCodemodExecution";
import { RepositoryModal } from "./RepositoryModal";
import { UserPromptModal } from "./UserPromptModal";

export const GHRunButton = memo(() => {
const { user } = useUser();
const router = useRouter();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@ export const RepositoryModal = ({
return isRepositoryModalShown ? (
<Modal onClose={hideRepositoryModal} centered transparent={false}>
<h2 className="text-center p-2">Run Codemod on Github branch</h2>

<DropdownSelector
isLoading={areReposLoading}
loadingMessage="Fetching repos"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { CodemodRunRequest } from "@shared/types";
import { useModStore } from "@studio/store/zustand/mod";
import { useSnippetStore } from "@studio/store/zustand/snippets";
import { useModStore } from "@studio/store/mod";
import { useSnippetsStore } from "@studio/store/snippets";
import { transpileTs } from "@studio/utils/transpileTs";
import type { GHBranch, GithubRepository } from "be-types";

Expand All @@ -14,15 +14,15 @@ export const useHandleCodemodRun = ({
selectedRepository,
selectedBranch,
}: Props) => {
const { engine } = useSnippetStore();
const { internalContent } = useModStore();
const isCodemodSourceNotEmpty = internalContent?.trim() !== "";
const { engine } = useSnippetsStore();
const { content } = useModStore();
const isCodemodSourceNotEmpty = content?.trim() !== "";

return async () => {
if (
selectedRepository === undefined ||
selectedBranch === undefined ||
internalContent === null ||
content === null ||
!isCodemodSourceNotEmpty ||
!(engine === "jscodeshift" || engine === "ts-morph") // other engines are not supported by backend API
) {
Expand All @@ -32,7 +32,7 @@ export const useHandleCodemodRun = ({
const request = {
codemodEngine: engine,
repoUrl: selectedRepository.html_url,
codemodSource: await transpileTs(internalContent),
codemodSource: await transpileTs(content),
branch: selectedBranch.name,
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { useAuth } from "@auth/useAuth";
import { GH_REPO_LIST } from "@shared/endpoints";
import { useAPI } from "@studio/hooks/useAPI";
import { useModal } from "@studio/hooks/useModal";
import { useUserSession } from "@studio/store/zustand/userSession";
import { useUserSession } from "@studio/store/utils/userSession";
import type { GithubRepository } from "be-types";
import { pipe } from "ramda";
import { type Dispatch, type SetStateAction, useEffect, useState } from "react";
Expand Down
Original file line number Diff line number Diff line change
@@ -1,34 +1,37 @@
import { useWebWorker } from "@/app/(website)/studio/features/codemod-apply/useWebWorker";
import { useCodemodOutputStore } from "@studio/store/zustand/codemodOutput";
import { useLogStore } from "@studio/store/zustand/log";
import { useModStore } from "@studio/store/zustand/mod";
import { useSnippetStore } from "@studio/store/zustand/snippets";
import { useLogStore } from "@studio/store/log";
import { useModStore } from "@studio/store/mod";
import { useSnippetsStore } from "@studio/store/snippets";
import { useEffect } from "react";

export const useCodemodOutputUpdate = () => {
const [webWorkerState, postMessage, setRetry] = useWebWorker();
const codemodOutput = useCodemodOutputStore();
const [webWorkerState, postMessage] = useWebWorker();
const { setEvents, events } = useLogStore();
const { setHasRuntimeErrors } = useModStore();
const { engine, inputSnippet } = useSnippetStore();
const { internalContent } = useModStore();
const snippetBeforeHasOnlyWhitespaces = !/\S/.test(inputSnippet);
const codemodSourceHasOnlyWhitespaces = !/\S/.test(internalContent ?? "");
const { setHasRuntimeErrors, content } = useModStore();
const {
engine,
getSelectedEditors,
currentContent,
currentType,
setSelectedPairIndex,
} = useSnippetsStore();
const { beforeSnippet, setOutputSnippet } = getSelectedEditors();
const snippetBeforeHasOnlyWhitespaces = !/\S/.test(beforeSnippet);
const codemodSourceHasOnlyWhitespaces = !/\S/.test(content ?? "");

useEffect(() => {
postMessage(engine, internalContent ?? "", inputSnippet);
setRetry(() => postMessage(engine, internalContent ?? "", inputSnippet));
postMessage(engine, content ?? "", beforeSnippet);
if (snippetBeforeHasOnlyWhitespaces || codemodSourceHasOnlyWhitespaces) {
codemodOutput.setContent("");
setOutputSnippet("");
setHasRuntimeErrors(false);
setEvents([]);
}
if (webWorkerState.kind === "LEFT") {
codemodOutput.setContent(webWorkerState.error.message);
setOutputSnippet(webWorkerState.error.message);
setHasRuntimeErrors(true);
setEvents([]);
} else {
codemodOutput.setContent(webWorkerState.output ?? "");
setOutputSnippet(webWorkerState.output ?? "");
setHasRuntimeErrors(true);
setEvents(webWorkerState.events);
}
Expand All @@ -39,11 +42,12 @@ export const useCodemodOutputUpdate = () => {
// @ts-ignore
webWorkerState.output,
engine,
inputSnippet,
internalContent,
snippetBeforeHasOnlyWhitespaces,
codemodSourceHasOnlyWhitespaces,
beforeSnippet,
currentContent,
currentType,
content,
postMessage,
setSelectedPairIndex,
]);

const firstCodemodExecutionErrorEvent = events.find(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,18 +158,18 @@ keys.forEach((key) => {
});

/**
Replaces: `$A.$B($C)`
Replaces: `$A.$B($C)`
with: `api.__method($A, $start, $end)($C);`
with: `api.__method($A, $start, $end)($C);`
The idea is that `$A` is likely an expression that ends up with a JSCodeshift collection.
The idea is that `$A` is likely an expression that ends up with a JSCodeshift collection.
The codemod runner tracks all the produced JSCodeshift collections.
In case of a mismatch, the runner will simply return `$A` from the `__method` call.
The codemod runner tracks all the produced JSCodeshift collections.
In case of a mismatch, the runner will simply return `$A` from the `__method` call.
`$start` and `$end` indicate the `$B` node position in the original codemod.
`$start` and `$end` indicate the `$B` node position in the original codemod.
This function will work with other JSCodeshift collection functions.
This function will work with other JSCodeshift collection functions.
**/
const replaceCallExpression = (
j: JSCodeshift,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
} from "@phosphor-icons/react";
import { Button } from "@studio/components/ui/button";
import { useCopyToClipboard } from "@studio/hooks/useCopyToClipboard";
import { useModStore } from "@studio/store/zustand/mod";
import { useModStore } from "@studio/store/mod";
import { prettify } from "@studio/utils/prettify";
import { type FC, memo } from "react";
import { Prism as SyntaxHighlighter } from "react-syntax-highlighter";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {
SelectTrigger,
SelectValue,
} from "@studio/components/ui/select";
import { useCFSStore } from "@studio/store/zustand/CFS";
import { useCFSStore } from "app/(website)/studio/src/store/CFS";

const legacyEngines = [
"gpt-4",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { ArrowElbowDownLeft } from "@phosphor-icons/react";
import { Button } from "@studio/components/ui/button";
import * as React from "react";
import type { PropsWithChildren } from "react";
import * as React from "react";

type PromptButtonsProps = {
promptsList: string[][];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import type { useAiService } from "@chatbot/useAiService/useAiService";
import type { useModGptSubmit } from "@chatbot/useAiService/useModGpt/useModGptSubmit";
import { getOrderedAliasList, usePrompts } from "@chatbot/utils";
import { useAuth } from "@clerk/nextjs";
import { useGetAliases } from "@studio/store/zustand/CFS/alias";
import { useGetAliases } from "@studio/store/CFS/alias";
import type { UseChatHelpers } from "ai/react";
import { useRef, useState } from "react";
import { PromptForm } from "./PromptForm";
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import type { useAiService } from "@chatbot/useAiService";
import type { useCodemodAI } from "@chatbot/useAiService/codemodAI/useCodemodAI";
import ButtonWithTooltip from "@studio/components/button/BottonWithTooltip";
import { Button } from "@studio/components/ui/button";
import Link from "next/link";

export const WebSocketButton = ({
Expand Down
4 changes: 3 additions & 1 deletion apps/frontend/app/(website)/studio/features/modGPT/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ export const isDevelopment = process.env.NODE_ENV === "development";
export const devToken =
"eyJhbGciOiJSUzI1NiIsImNhdCI6ImNsX0I3ZDRQRDIyMkFBQSIsImtpZCI6Imluc18yTkhTQmFySFpONDRlVm5PNTVoM1pSbmdNSUUiLCJ0eXAiOiJKV1QifQ.eyJhenAiOiJodHRwczovL2NvZGVtb2QuY29tIiwiZXhwIjoyMDMyMTg2NjI3LCJpYXQiOjE3MTY4MjY2MjcsImlzcyI6Imh0dHBzOi8vY2xlcmsuY29kZW1vZC5jb20iLCJqdGkiOiIyNzc3NDcxZmE2NzBkZGYzMGY1MiIsIm5iZiI6MTcxNjgyNjYyMiwic3ViIjoidXNlcl8yZFkzczltb2JVQjhJZTRVOGg2dkF2YUtBNzMifQ.HButODofVbhyZbKZD14QGIwOWI3nMubCkjoNB-V1uONiqhdMsi3ZebPQAsRKss5-jnEYWg_YZ1c5jZf50iewLvg8h9Pr3Hd0srech98MPon8zCuaYlbE2Hs0poVS94mHXNfN8qCb5wm1GQ-ZM-l1Ux3yJtJ_Ge-hL-GIKHEo11FusTCPZzdMxVJEZXL454sQ1DRhmVMwCmjybzMt4yB-AQL77ieWMxkynyYdI8MhIhmqqlSR-_17_jtdAvmDH5Z4lgr3q0bcEluxwclIZAMSlLJ_mZkQCxnjwgm1Z5kYhE-fO7xfIogX3lX3aolpQczMPIHQXFSqDtMlxonFKt0iHg";

export const codemodAiWsServer = "wss://backend.codemod.com/ws";
export const codemodAiWsServer = isDevelopment
? "ws://127.0.0.1:8000/ws"
: "wss://backend.codemod.com/ws";

const prodGptServer = "https://backend.codemod.com/modgpt";

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { useAuth } from "@/app/auth/useAuth";
import { codemodAiWsServer } from "@chatbot/config";
import type { LLMMessage, MessageFromWs, MessageToWs } from "@chatbot/types";
import type { LLMEngine } from "@shared/consts";
import { useSnippetStore } from "@studio/store/zustand/snippets";
import { useSnippetsStore } from "@studio/store/snippets";
import type { ToVoid } from "@studio/types/transformations";
import { useEffect, useState } from "react";
import toast from "react-hot-toast";
Expand All @@ -11,8 +11,8 @@ import { type Socket, io } from "socket.io-client";
type MessageToSend = {
config: { llm_engine: LLMEngine };
previous_context: LLMMessage[];
before: string;
after: string;
before: string | string[];
after: string | string[];
};
export const useCodemodAI = ({
setToken,
Expand All @@ -25,15 +25,14 @@ export const useCodemodAI = ({
}) => {
const [ws, setWs] = useState<WebSocket | null>(null);
const [wsMessage, setWsMessage] = useState<MessageFromWs>();
const { inputSnippet: before, afterSnippet: after } = useSnippetStore();
const { getAllSnippets } = useSnippetsStore();
const [isWsConnected, setIsWsConnected] = useState(false);
const [serviceBusy, setServiceBusy] = useState(false);
const { getToken } = useAuth();
const emitMessage = async (message: MessageToSend) => {
const _token = await getToken();
setToken(_token);
ws?.send(JSON.stringify({ ...message, token: _token }));
// socket?.emit("message", message);
};
const handleError = (error: Record<string, unknown> | Event) => {
setServiceBusy(false);
Expand Down Expand Up @@ -108,8 +107,16 @@ export const useCodemodAI = ({
return () => cleanup();
}, []);

const beforeSnippets = getAllSnippets().before;
const afterSnippets = getAllSnippets().after;
const startIterativeCodemodGeneration = async () => {
if (ws && before && after && isWsConnected && !serviceBusy) {
if (
ws &&
beforeSnippets.length &&
afterSnippets.length &&
isWsConnected &&
!serviceBusy
) {
const _token = await getToken();
setToken(_token);
setWsMessage({
Expand All @@ -120,8 +127,8 @@ export const useCodemodAI = ({
const messageToSend: MessageToSend = {
config: { llm_engine: engine },
previous_context: messages,
before,
after,
before: beforeSnippets,
after: afterSnippets,
};
emitMessage(messageToSend);
setServiceBusy(true);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { modGptServer } from "@chatbot/config";
import { useModGptSubmit } from "@chatbot/useAiService/useModGpt/useModGptSubmit";
import { onResponse } from "@chatbot/utils";
import type { LLMEngine } from "@shared/consts";
import { useModStore } from "@studio/store/zustand/mod";
import { useModStore } from "@studio/store/mod";
import type { Message } from "ai";
import { useChat } from "ai/react";
import { useCallback, useState } from "react";
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { useAuth } from "@auth/useAuth";
import { getHeadersWithAuth } from "@chatbot/useAiService/useModGpt/utils";
import { applyAliases, useGetAliases } from "@studio/store/zustand/CFS/alias";
import { applyAliases, useGetAliases } from "@studio/store/CFS/alias";
import type { UseChatHelpers } from "ai/react/dist";
import type { Dispatch, SetStateAction } from "react";

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { useAuth } from "@auth/useAuth";
import { autoGenerateCodemodPrompt } from "@chatbot/prompts";
import { applyAliases, useGetAliases } from "@studio/store/zustand/CFS/alias";
import { useModStore } from "@studio/store/zustand/mod";
import { applyAliases, useGetAliases } from "@studio/store/CFS/alias";
import { useModStore } from "@studio/store/mod";
import type { useChat } from "ai/react/dist";
import { identity } from "ramda";
import { type Dispatch, type SetStateAction, useEffect, useRef } from "react";
Expand Down
4 changes: 2 additions & 2 deletions apps/frontend/app/(website)/studio/features/modGPT/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ import {
autoGenerateCodemodPrompt,
fixCodemodBlockNoDebugInfoPrompt,
} from "@chatbot/prompts";
import type { Aliases } from "@studio/store/zustand/CFS/alias";
import { useCodemodExecutionError } from "@studio/store/zustand/log";
import type { Aliases } from "@studio/store/CFS/alias";
import { useCodemodExecutionError } from "@studio/store/log";
import toast from "react-hot-toast";

const errorResponses = {
Expand Down
Loading
Loading