Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ai rule gen #2158

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2df17df
First version of the AI auto rule gen endpoint
mirrormystic Sep 24, 2024
db10c88
Working on the AI gen table
mirrormystic Sep 27, 2024
d29abb2
Working on the AI gen table
mirrormystic Sep 27, 2024
252bbdd
More work on gen rules
mirrormystic Sep 27, 2024
1a82fa4
Fixing tooltips, fixing <aigen> component loading
mirrormystic Sep 29, 2024
e05f277
Adding things to tiktoken lib to poerty, making the rules api send on…
mirrormystic Sep 29, 2024
2a04e6d
genAIRules now using fetcher
mirrormystic Oct 6, 2024
4464fac
fixed fetcher bugs
mirrormystic Oct 8, 2024
9c151a3
verifying CEL rules returned from the llm
mirrormystic Oct 9, 2024
48796fb
Merge remote-tracking branch 'origin/main' into feature/1968-ai-rule-gen
mirrormystic Oct 9, 2024
f46fa61
fixing perms check, and a small amount of estetic cleanups
mirrormystic Oct 9, 2024
6af8318
Fixing some warnings
mirrormystic Oct 9, 2024
8275b63
Removing bear except
mirrormystic Oct 9, 2024
43175d7
adding lock file
mirrormystic Oct 9, 2024
482bc2e
removing unused params
mirrormystic Oct 9, 2024
69d6e29
fixing js errors
mirrormystic Oct 9, 2024
11bdd09
fixing ts error
mirrormystic Oct 10, 2024
3994bbe
changing up the rule table
mirrormystic Oct 10, 2024
b398396
adding an import
mirrormystic Oct 10, 2024
1579271
pretting code
mirrormystic Oct 10, 2024
1a75a2c
Merge branch 'main' into feature/1968-ai-rule-gen
Kiryous Oct 13, 2024
0de2304
fixing CR issues, showing error msgs and the right loader
mirrormystic Oct 13, 2024
23a6379
merging
mirrormystic Oct 13, 2024
786fd8f
Merge branch 'main' of github.com:keephq/keep into feature/1968-ai-ru…
mirrormystic Oct 13, 2024
95a206b
Merge branch 'main' into feature/1968-ai-rule-gen
Kiryous Oct 13, 2024
cf4d281
Merge branch 'main' into feature/1968-ai-rule-gen
Kiryous Oct 14, 2024
73a7fb5
Changing up the AI rules thing
mirrormystic Oct 14, 2024
e5ce5ad
fixing up rule gen UI
mirrormystic Oct 14, 2024
d5d85e6
fixing compile error
mirrormystic Oct 14, 2024
7c17e2a
cleaning up stuff
mirrormystic Oct 14, 2024
4e08816
changing the loading text
mirrormystic Oct 14, 2024
7ef3a6f
fix: the tabs height
Kiryous Oct 14, 2024
64fc3d5
fix: fix
talboren Oct 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions keep-ui/app/loading.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ import Image from "next/image";
export default function Loading({
includeMinHeight = true,
slowLoading = false,
loadingText = "Just a second, getting your data 🚨"
}: {
includeMinHeight?: boolean;
slowLoading?: boolean;
}) {
loadingText?: string;
}) {
return (
<main
className={`flex flex-col items-center justify-center ${
Expand All @@ -21,7 +23,7 @@ export default function Loading({
width={200}
height={200}
/>
<Title>Just a second, getting your data 🚨</Title>
<Title>{loadingText}</Title>
{slowLoading && (
<Subtitle>
This is taking a bit longer then usual, please wait...
Expand Down
318 changes: 318 additions & 0 deletions keep-ui/app/rules/AIGenRules.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,318 @@
import React, { useState, useEffect } from 'react';
import { Icon, Table, TableBody, TableCell, TableHead, TableHeaderCell, TableRow } from "@tremor/react";
import {
createColumnHelper,
getCoreRowModel,
useReactTable,
ColumnDef,
getSortedRowModel,
SortingState,
flexRender,
Header
} from "@tanstack/react-table";
import { useRulePusherUpdates, AIGeneratedRule, useGenRules } from "utils/hooks/useRules";
import { FaArrowDown, FaArrowRight, FaArrowUp, FaPlus, FaSpinner, FaSync } from "react-icons/fa";
import { InformationCircleIcon, ExclamationTriangleIcon, QuestionMarkCircleIcon } from "@heroicons/react/24/solid";
import { useSession } from "next-auth/react";
import { getApiURL } from "utils/apiUrl";
import useSWR, { mutate } from "swr";
import Loading from "app/loading";



const columnHelper = createColumnHelper<AIGeneratedRule>();

interface SortableHeaderCellProps {
header: Header<AIGeneratedRule, unknown>;
children: React.ReactNode;
}

const SortableHeaderCell: React.FC<SortableHeaderCellProps> = ({
header,
children,
}) => {
const { column } = header;

return (
<TableHeaderCell
className={`relative ${column.getIsPinned() ? "" : "hover:bg-slate-100"} group`}
>
<div className="flex items-center">
{children} {/* Column name or text */}
{column.getCanSort() && (
<>
{/* Custom styled vertical line separator */}
<div className="w-px h-5 mx-2 bg-gray-400"></div>
<Icon
className="cursor-pointer"
size="xs"
color="neutral"
onClick={(event) => {
event.stopPropagation();
const toggleSorting = header.column.getToggleSortingHandler();
if (toggleSorting) toggleSorting(event);
}}
tooltip={
column.getNextSortingOrder() === "asc"
? "Sort ascending"
: column.getNextSortingOrder() === "desc"
? "Sort descending"
: "Clear sort"
}
icon={column.getIsSorted() ? (
column.getIsSorted() === "asc" ? FaArrowDown : FaArrowUp
) : (
FaArrowRight
)}
/>
</>
)}
</div>
</TableHeaderCell>
);
};

export const AIGenRules: React.FC = () => {
const {triggerGenRules } = useGenRules();
const { data: session } = useSession();
const [sorting, setSorting] = React.useState<SortingState>([]);
mirrormystic marked this conversation as resolved.
Show resolved Hide resolved

const [isLoadingRules, setIsLoadingRules] = useState(true);
const [generatedRules, setGeneratedRules] = useState<any>([]);

const [loadingRows, setLoadingRows] = useState<{ [key: string]: boolean }>({});
const [successRows, setSuccessRows] = useState<{ [key: string]: boolean }>({});

const mutateAIGeneratedRules = (rules: AIGeneratedRule[]) => {
setGeneratedRules(rules);
setIsLoadingRules(false);
};

const { data:serverGenRules } = useRulePusherUpdates();

useEffect(() => {
console.log("serverGenRules", serverGenRules);
mirrormystic marked this conversation as resolved.
Show resolved Hide resolved

if (Array.isArray(serverGenRules) && (0 === serverGenRules.length)) {
return;
}

mutateAIGeneratedRules(serverGenRules);
}, [serverGenRules]);

const handleGenerateMoreRules = () => {
setIsLoadingRules(true);
triggerGenRules();
};

const handleAddRule = async (rule: AIGeneratedRule) => {
mirrormystic marked this conversation as resolved.
Show resolved Hide resolved
const ruleKey = rule.ShortRuleName;
setLoadingRows((prev) => ({ ...prev, [ruleKey]: true }));
try {
const temp = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${session?.accessToken}`,
},
body: JSON.stringify({
ruleName: rule.ShortRuleName,
sqlQuery: { sql: '{new-version-not-adding-this}', params: ['no-params'] },
celQuery: rule.CELRule,
timeframeInSeconds: rule.Timeframe,
timeUnit: 'minutes',
groupingCriteria: rule.GroupBy,
groupDescription: rule.ChainOfThought,
requireApprove: false,
}),
};

const apiUrl = getApiURL();
const response = await fetch(`${apiUrl}/rules`, temp);

if (response.ok) {
setSuccessRows((prev) => ({ ...prev, [ruleKey]: true }));
mutate(`${apiUrl}/rules`); // Refresh the rules list
} else {
const result = await response.json();
console.error('Failed to add rule:', result);
}
} catch (error) {
console.error('Error adding rule:', error);
} finally {
setLoadingRows((prev) => {
const newLoadingRows = { ...prev };
delete newLoadingRows[ruleKey];
return newLoadingRows;
});
}
};

const columns = React.useMemo(() => [
columnHelper.accessor("ShortRuleName", {
header: "Short Rule Name",
}),
columnHelper.accessor("Score", {
header: "Score",
}),
columnHelper.accessor("CELRule", {
header: "CEL Rule",
cell: (info) => <div className="text-wrap">{info.getValue()}</div>,
}),
columnHelper.accessor("Timeframe", {
header: "Timeframe",
}),
columnHelper.accessor("GroupBy", {
header: "Group By",
cell: (info) => info.getValue().join(", "),
}),
columnHelper.accessor("ChainOfThought", {
header: "Explanations",
cell: (info) => {
const rule = info.row.original;
return (
<div className="flex space-x-2">
<Icon
mirrormystic marked this conversation as resolved.
Show resolved Hide resolved
icon={InformationCircleIcon}
tooltip={`Thinking behind the rule: ${rule.ChainOfThought}`}
size="xs"
color="gray"
className="ml-1"
variant="solid"
/>
<Icon
icon={ExclamationTriangleIcon}
tooltip={`Why this rule is too general: ${rule.WhyTooGeneral}`}
size="xs"
color="gray"
className="ml-1"
variant="solid"
/>
<Icon
icon={QuestionMarkCircleIcon}
tooltip={`Why this rule is too specific: ${rule.WhyTooSpecific}`}
size="xs"
color="gray"
className="ml-1"
variant="solid"
/>
</div>
);
},
}),
columnHelper.display({
id: 'add',
header: 'Add',
cell: (info) => {
const rule = info.row.original;
const ruleKey = rule.ShortRuleName;
return (
<div className="flex justify-center items-center">
{loadingRows[ruleKey] ? (
<FaSpinner className="animate-spin text-gray-500" />
) : successRows[ruleKey] ? (
<div className="text-green-500">Added!</div>
) : (
<button
onClick={() => handleAddRule(rule)}
className="text-blue-500 hover:text-blue-700"
>
<FaPlus />
</button>
)}
</div>
);
},
}),
] as ColumnDef<AIGeneratedRule>[], [loadingRows, successRows]);

const table = useReactTable({
columns,
data: generatedRules?.results || [],
getCoreRowModel: getCoreRowModel(),
getSortedRowModel: getSortedRowModel(),
onSortingChange: setSorting,
state: {
sorting,
},
});

if (isLoadingRules) {
return <Loading loadingText="Generating AI recommendations..." />;
mirrormystic marked this conversation as resolved.
Show resolved Hide resolved
}

if (generatedRules.error) {
return (
<div className="flex justify-center items-center h-64">
<div className="text-center">
<p className="text-lg font-semibold text-red-600">{generatedRules.error}</p>
<p className="text-sm text-gray-500">Please try again later</p>
</div>
</div>
);
}


return (
<div className="space-y-4">
<div className="flex flex-col items-start">
<button
onClick={handleGenerateMoreRules}
className="px-4 py-2 bg-orange-500 text-white rounded hover:bg-orange-600 flex items-center"
>
Generate more rules
</button>
<h2 className="text-xl font-semibold mt-4 self-center">AI Generated Rules</h2>
</div>
<p className="text-center">{generatedRules.summery}</p>
<Table>
<TableHead>
{table.getHeaderGroups().map((headerGroup) => (
<TableRow
className="border-b border-tremor-border dark:border-dark-tremor-border"
key={headerGroup.id}
>
{headerGroup.headers.map((header) => (
<SortableHeaderCell
header={header}
key={header.id}
>
{flexRender(
header.column.columnDef.header,
header.getContext()
)}
</SortableHeaderCell>
))}
</TableRow>
))}
</TableHead>
<TableBody>
{table.getRowModel().rows.map((row) => {
const rule = row.original;
const ruleKey = rule.ShortRuleName;
return (
<TableRow
className={`${
successRows[ruleKey]
? 'bg-green-100'
: 'even:bg-tremor-background-muted even:dark:bg-dark-tremor-background-muted hover:bg-slate-100'
} cursor-pointer`}
key={row.id}
>
{row.getVisibleCells().map((cell) => (
<TableCell key={cell.id}>
{flexRender(cell.column.columnDef.cell, cell.getContext())}
</TableCell>
))}
</TableRow>
);
})}
</TableBody>
</Table>
</div>
);
};

export default AIGenRules;


7 changes: 0 additions & 7 deletions keep-ui/app/rules/CorrelationPlaceholder.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,6 @@ export const CorrelationPlaceholder = () => {
dedicated place
</Subtitle>
</div>
<Button
className="mb-10"
color="orange"
onClick={() => onCorrelationClick()}
>
Create Correlation
</Button>
<PlaceholderSankey className="max-w-full" />
</Card>
<CorrelationSidebar
Expand Down
2 changes: 1 addition & 1 deletion keep-ui/app/rules/CorrelationSidebar/DeleteRule.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ export const DeleteRuleCell = ({ ruleId }: DeleteRuleCellProps) => {

return (
<Button
className="invisible group-hover:visible"
className=""
mirrormystic marked this conversation as resolved.
Show resolved Hide resolved
onClick={onDeleteRule}
variant="secondary"
color="red"
Expand Down
Loading
Loading