-
-
Notifications
You must be signed in to change notification settings - Fork 82
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add streaming upload example. (#206)
- Loading branch information
1 parent
9170a2a
commit 11c4b66
Showing
5 changed files
with
219 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
source 'https://rubygems.org' | ||
|
||
gem "falcon" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
GEM | ||
remote: https://rubygems.org/ | ||
specs: | ||
async (2.5.0) | ||
console (~> 1.10) | ||
io-event (~> 1.1) | ||
timers (~> 4.1) | ||
async-container (0.16.12) | ||
async | ||
async-io | ||
async-http (0.60.1) | ||
async (>= 1.25) | ||
async-io (>= 1.28) | ||
async-pool (>= 0.2) | ||
protocol-http (~> 0.24.0) | ||
protocol-http1 (~> 0.15.0) | ||
protocol-http2 (~> 0.15.0) | ||
traces (>= 0.8.0) | ||
async-http-cache (0.4.3) | ||
async-http (~> 0.56) | ||
async-io (1.34.3) | ||
async | ||
async-pool (0.4.0) | ||
async (>= 1.25) | ||
build-environment (1.13.0) | ||
console (1.16.2) | ||
fiber-local | ||
falcon (0.42.3) | ||
async | ||
async-container (~> 0.16.0) | ||
async-http (~> 0.57) | ||
async-http-cache (~> 0.4.0) | ||
async-io (~> 1.22) | ||
build-environment (~> 1.13) | ||
bundler | ||
localhost (~> 1.1) | ||
openssl (~> 3.0) | ||
process-metrics (~> 0.2.0) | ||
protocol-rack (~> 0.1) | ||
samovar (~> 2.1) | ||
fiber-local (1.0.0) | ||
io-event (1.1.7) | ||
localhost (1.1.10) | ||
mapping (1.1.1) | ||
openssl (3.1.0) | ||
process-metrics (0.2.1) | ||
console (~> 1.8) | ||
samovar (~> 2.1) | ||
protocol-hpack (1.4.2) | ||
protocol-http (0.24.1) | ||
protocol-http1 (0.15.0) | ||
protocol-http (~> 0.22) | ||
protocol-http2 (0.15.1) | ||
protocol-hpack (~> 1.4) | ||
protocol-http (~> 0.18) | ||
protocol-rack (0.2.4) | ||
protocol-http (~> 0.23) | ||
rack (>= 1.0) | ||
rack (3.0.7) | ||
samovar (2.1.4) | ||
console (~> 1.0) | ||
mapping (~> 1.0) | ||
timers (4.3.5) | ||
traces (0.9.1) | ||
|
||
PLATFORMS | ||
x86_64-linux | ||
|
||
DEPENDENCIES | ||
falcon | ||
|
||
BUNDLED WITH | ||
2.4.6 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# Streaming Upload | ||
|
||
# ab Results | ||
|
||
The filesize was 82078050 bytes. | ||
|
||
|
||
|
||
|
||
## concurrent requests | ||
|
||
``` | ||
bundle exec falcon host ./falcon.rb | ||
``` | ||
|
||
|
||
``` | ||
This is ApacheBench, Version 2.3 <$Revision: 1879490 $> | ||
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ | ||
Licensed to The Apache Software Foundation, http://www.apache.org/ | ||
|
||
Benchmarking localhost (be patient).....done | ||
|
||
|
||
Server Software: | ||
Server Hostname: localhost | ||
Server Port: 9292 | ||
|
||
Document Path: / | ||
Document Length: 100 bytes | ||
|
||
Concurrency Level: 20 | ||
Time taken for tests: 7.873 seconds | ||
Complete requests: 100 | ||
Failed requests: 9 | ||
(Connect: 0, Receive: 0, Length: 9, Exceptions: 0) | ||
Total transferred: 16280 bytes | ||
Total body sent: 8628660682 | ||
HTML transferred: 9989 bytes | ||
Requests per second: 12.70 [#/sec] (mean) | ||
Time per request: 1574.576 [ms] (mean) | ||
Time per request: 78.729 [ms] (mean, across all concurrent requests) | ||
Transfer rate: 2.02 [Kbytes/sec] received | ||
1070310.40 kb/s sent | ||
1070312.42 kb/s total | ||
|
||
Connection Times (ms) | ||
min mean[+/-sd] median max | ||
Connect: 0 56 44.6 52 141 | ||
Processing: 296 1384 725.4 1159 4456 | ||
Waiting: 33 126 89.2 121 886 | ||
Total: 297 1440 720.8 1236 4464 | ||
|
||
Percentage of the requests served within a certain time (ms) | ||
50% 1236 | ||
66% 1458 | ||
75% 1627 | ||
80% 1711 | ||
90% 2020 | ||
95% 3883 | ||
98% 4463 | ||
99% 4464 | ||
100% 4464 (longest request) | ||
``` | ||
|
||
EOFError errors cause the failed uploads....have to figure out why. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# frozen_string_literal: true | ||
|
||
# run with: falcon serve -n 1 -b http://localhost:9292 | ||
|
||
require 'digest' | ||
require 'securerandom' | ||
|
||
module Protocol | ||
module HTTP1 | ||
class Connection | ||
def write_continue | ||
@stream.write("HTTP/1.1 100 Continue\r\n\r\n") | ||
@stream.flush | ||
end | ||
end | ||
end | ||
end | ||
|
||
class BodyHandler | ||
attr_reader :time, :md5, :size, :uuid | ||
|
||
def initialize(input, length) | ||
@uuid = SecureRandom.uuid | ||
@input = input | ||
@length = length | ||
end | ||
|
||
def receive | ||
start = Time.now | ||
@md5 = Digest::MD5.new | ||
@size = 0 | ||
@done = false | ||
|
||
until @done | ||
begin | ||
chunk = @input.read(10_240) # read will raise EOF so we have to check | ||
rescue EOFError | ||
puts "Seems we're done" | ||
chunk = nil | ||
end | ||
if chunk.nil? | ||
@done = true | ||
else | ||
@md5.update(chunk) | ||
@size += chunk.bytesize | ||
@done = true if @length == @size | ||
end | ||
end | ||
|
||
@time = Time.now - start | ||
end | ||
end | ||
|
||
run lambda { |env| | ||
request = env['protocol.http.request'] | ||
handler = BodyHandler.new(env['rack.input'], env['CONTENT_LENGTH']) | ||
puts "#{env['REQUEST_METHOD']} #{handler.uuid}: #{request.path} #{env['CONTENT_LENGTH']}" | ||
|
||
if env['REQUEST_METHOD'] == 'POST' | ||
request.connection.write_continue if request.headers['expect'] == ['100-continue'] | ||
handler.receive | ||
msg = "Uploaded #{handler.uuid}: #{handler.md5} #{handler.time} #{handler.size}" | ||
puts msg | ||
[200, {}, [msg]] | ||
else | ||
sleep 1 | ||
[200, {}, ["#{env['REQUEST_METHOD']}: #{request.path}\n"]] | ||
end | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
load :rack, :supervisor | ||
|
||
hostname = File.basename(__dir__) | ||
rack hostname do | ||
endpoint Async::HTTP::Endpoint.parse('http://localhost:9292').with(protocol: Async::HTTP::Protocol::HTTP1) | ||
end | ||
|
||
supervisor |