Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add AWS SDK v3 upload example #164

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions file-and-s3-upload-v2/.env.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
STORAGE_ACCESS_KEY=
STORAGE_SECRET=
STORAGE_REGION=
STORAGE_BUCKET=
4 changes: 4 additions & 0 deletions file-and-s3-upload-v2/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/** @type {import('eslint').Linter.Config} */
module.exports = {
extends: ["@remix-run/eslint-config", "@remix-run/eslint-config/node"],
};
6 changes: 6 additions & 0 deletions file-and-s3-upload-v2/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
node_modules

/.cache
/build
/public/build
.env
37 changes: 37 additions & 0 deletions file-and-s3-upload-v2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Upload images to S3 with AWS SDK v2

> **Note:** This example is for the old AWS SDK v2. For the updated with the AWS SDK v3 example, see [file-and-s3-upload](https://github.com/remix-run/examples/tree/main/file-and-s3-upload).

This is a simple example of using the remix built-in [uploadHandler](https://remix.run/utils/parse-multipart-form-data#uploadhandler) and Form with multipart data to upload a file with the built-in local uploader and upload an image file to S3 with a custom uploader and display it. You can test it locally by running the dev server and opening the path `/s3-upload` in your browser.

The relevent files are:

```
├── app
│ ├── routes
│ │ ├── s3-upload.tsx // upload to S3
│ └── utils
│ └── s3.server.ts // init S3 client on server side
|── .env // holds AWS S3 credentails
```

## Steps to set up an S3 bucket

- Sign up for an [AWS account](https://portal.aws.amazon.com/billing/signup) - this will require a credit card
- Create an S3 bucket in your desired region
- Create an access key pair for an IAM user that has access to the bucket
- Copy the .env.sample to .env and fill in the S3 bucket, the region as well as the access key and secret key from the IAM user

Note: in order for the image to be displayed after being uploaded to your S3 bucket in this example, the bucket needs to have public access enabled, which is potentially dangerous.

> :warning: Lambda imposes a [limit of 6MB](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html) on the invocation payload size. If you use this example with Remix running on Lambda, you can only update files with a size smaller than 6MB.

Open this example on [CodeSandbox](https://codesandbox.com):

[![Open in CodeSandbox](https://codesandbox.io/static/img/play-codesandbox.svg)](https://codesandbox.io/s/github/remix-run/examples/tree/main/file-and-s3-upload)

## Related Links

- [Handle Multiple Part Forms (File Uploads)](https://remix.run/utils/parse-multipart-form-data-node)
- [Upload Handler](https://remix.run/utils/parse-multipart-form-data#uploadhandler)
- [Custom Uploader](https://remix.run/guides/file-uploads)
21 changes: 21 additions & 0 deletions file-and-s3-upload-v2/app/entry.client.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import { RemixBrowser } from "@remix-run/react";
import { startTransition, StrictMode } from "react";
import { hydrateRoot } from "react-dom/client";

const hydrate = () =>
startTransition(() => {
hydrateRoot(
document,
<StrictMode>
<RemixBrowser />
</StrictMode>
);
});

if (typeof requestIdleCallback === "function") {
requestIdleCallback(hydrate);
} else {
// Safari doesn't support requestIdleCallback
// https://caniuse.com/requestidlecallback
setTimeout(hydrate, 1);
}
110 changes: 110 additions & 0 deletions file-and-s3-upload-v2/app/entry.server.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
import { PassThrough } from "stream";

import type { EntryContext } from "@remix-run/node";
import { Response } from "@remix-run/node";
import { RemixServer } from "@remix-run/react";
import isbot from "isbot";
import { renderToPipeableStream } from "react-dom/server";

const ABORT_DELAY = 5000;

const handleRequest = (
request: Request,
responseStatusCode: number,
responseHeaders: Headers,
remixContext: EntryContext
) =>
isbot(request.headers.get("user-agent"))
? handleBotRequest(
request,
responseStatusCode,
responseHeaders,
remixContext
)
: handleBrowserRequest(
request,
responseStatusCode,
responseHeaders,
remixContext
);
export default handleRequest;

const handleBotRequest = (
request: Request,
responseStatusCode: number,
responseHeaders: Headers,
remixContext: EntryContext
) =>
new Promise((resolve, reject) => {
let didError = false;

const { pipe, abort } = renderToPipeableStream(
<RemixServer context={remixContext} url={request.url} />,
{
onAllReady: () => {
const body = new PassThrough();

responseHeaders.set("Content-Type", "text/html");

resolve(
new Response(body, {
headers: responseHeaders,
status: didError ? 500 : responseStatusCode,
})
);

pipe(body);
},
onShellError: (error: unknown) => {
reject(error);
},
onError: (error: unknown) => {
didError = true;

console.error(error);
},
}
);

setTimeout(abort, ABORT_DELAY);
});

const handleBrowserRequest = (
request: Request,
responseStatusCode: number,
responseHeaders: Headers,
remixContext: EntryContext
) =>
new Promise((resolve, reject) => {
let didError = false;

const { pipe, abort } = renderToPipeableStream(
<RemixServer context={remixContext} url={request.url} />,
{
onShellReady: () => {
const body = new PassThrough();

responseHeaders.set("Content-Type", "text/html");

resolve(
new Response(body, {
headers: responseHeaders,
status: didError ? 500 : responseStatusCode,
})
);

pipe(body);
},
onShellError: (error: unknown) => {
reject(error);
},
onError: (error: unknown) => {
didError = true;

console.error(error);
},
}
);

setTimeout(abort, ABORT_DELAY);
});
32 changes: 32 additions & 0 deletions file-and-s3-upload-v2/app/root.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import type { MetaFunction } from "@remix-run/node";
import {
Links,
LiveReload,
Meta,
Outlet,
Scripts,
ScrollRestoration,
} from "@remix-run/react";

export const meta: MetaFunction = () => ({
charset: "utf-8",
title: "New Remix App",
viewport: "width=device-width,initial-scale=1",
});

export default function App() {
return (
<html lang="en">
<head>
<Meta />
<Links />
</head>
<body>
<Outlet />
<ScrollRestoration />
<Scripts />
<LiveReload />
</body>
</html>
);
}
68 changes: 68 additions & 0 deletions file-and-s3-upload-v2/app/routes/s3-upload.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import type { ActionArgs, UploadHandler } from "@remix-run/node";
import {
json,
unstable_composeUploadHandlers as composeUploadHandlers,
unstable_createMemoryUploadHandler as createMemoryUploadHandler,
unstable_parseMultipartFormData as parseMultipartFormData,
} from "@remix-run/node";
import { useFetcher } from "@remix-run/react";

import { s3UploadHandler } from "~/utils/s3.server";

type ActionData = {
errorMsg?: string;
imgSrc?: string;
imgDesc?: string;
};

export const action = async ({ request }: ActionArgs) => {
const uploadHandler: UploadHandler = composeUploadHandlers(
s3UploadHandler,
createMemoryUploadHandler()
);
const formData = await parseMultipartFormData(request, uploadHandler);
const imgSrc = formData.get("img");
const imgDesc = formData.get("desc");
console.log(imgDesc);
if (!imgSrc) {
return json({
errorMsg: "Something went wrong while uploading",
});
}
return json({
imgSrc,
imgDesc,
});
};

export default function Index() {
const fetcher = useFetcher<ActionData>();
return (
<>
<fetcher.Form method="post" encType="multipart/form-data">
<label htmlFor="img-field">Image to upload</label>
<input id="img-field" type="file" name="img" accept="image/*" />
<label htmlFor="img-desc">Image description</label>
<input id="img-desc" type="text" name="desc" />
<button type="submit">Upload to S3</button>
</fetcher.Form>
{fetcher.type === "done" ? (
fetcher.data.errorMsg ? (
<h2>{fetcher.data.errorMsg}</h2>
) : (
<>
<div>
File has been uploaded to S3 and is available under the following
URL (if the bucket has public access enabled):
</div>
<div>{fetcher.data.imgSrc}</div>
<img
src={fetcher.data.imgSrc}
alt={fetcher.data.imgDesc || "Uploaded image from S3"}
/>
</>
)
) : null}
</>
);
}
50 changes: 50 additions & 0 deletions file-and-s3-upload-v2/app/utils/s3.server.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import { PassThrough } from "stream";

import AWS from "aws-sdk";
import type { UploadHandler } from "@remix-run/node";
import { writeAsyncIterableToWritable } from "@remix-run/node";

const { STORAGE_ACCESS_KEY, STORAGE_SECRET, STORAGE_REGION, STORAGE_BUCKET } =
process.env;

if (
!(STORAGE_ACCESS_KEY && STORAGE_SECRET && STORAGE_REGION && STORAGE_BUCKET)
) {
throw new Error(`Storage is missing required configuration.`);
}

const uploadStream = ({ Key }: Pick<AWS.S3.Types.PutObjectRequest, "Key">) => {
const s3 = new AWS.S3({
credentials: {
accessKeyId: STORAGE_ACCESS_KEY,
secretAccessKey: STORAGE_SECRET,
},
region: STORAGE_REGION,
});
const pass = new PassThrough();
return {
writeStream: pass,
promise: s3.upload({ Bucket: STORAGE_BUCKET, Key, Body: pass }).promise(),
};
};

export async function uploadStreamToS3(data: any, filename: string) {
const stream = uploadStream({
Key: filename,
});
await writeAsyncIterableToWritable(data, stream.writeStream);
const file = await stream.promise;
return file.Location;
}

export const s3UploadHandler: UploadHandler = async ({
name,
filename,
data,
}) => {
if (name !== "img") {
return undefined;
}
const uploadedFileLocation = await uploadStreamToS3(data, filename!);
return uploadedFileLocation;
};
30 changes: 30 additions & 0 deletions file-and-s3-upload-v2/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"private": true,
"sideEffects": false,
"scripts": {
"build": "remix build",
"dev": "remix dev",
"start": "remix-serve build",
"typecheck": "tsc"
},
"dependencies": {
"@remix-run/node": "*",
"@remix-run/react": "*",
"@remix-run/serve": "*",
"aws-sdk": "^2.1152.0",
"isbot": "^3.6.5",
"react": "^18.2.0",
"react-dom": "^18.2.0"
},
"devDependencies": {
"@remix-run/dev": "*",
"@remix-run/eslint-config": "*",
"@types/react": "^18.0.25",
"@types/react-dom": "^18.0.8",
"eslint": "^8.27.0",
"typescript": "^4.6.2"
},
"engines": {
"node": ">=14"
}
}
Binary file added file-and-s3-upload-v2/public/favicon.ico
Binary file not shown.
10 changes: 10 additions & 0 deletions file-and-s3-upload-v2/remix.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/**
* @type {import('@remix-run/dev').AppConfig}
*/
module.exports = {
ignoredRouteFiles: ["**/.*"],
// appDirectory: "app",
// assetsBuildDirectory: "public/build",
// serverBuildPath: "build/index.js",
// publicPath: "/build/",
};
2 changes: 2 additions & 0 deletions file-and-s3-upload-v2/remix.env.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/// <reference types="@remix-run/dev" />
/// <reference types="@remix-run/node" />
7 changes: 7 additions & 0 deletions file-and-s3-upload-v2/sandbox.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"hardReloadOnChange": true,
"template": "remix",
"container": {
"port": 3000
}
}
Loading