Skip to content

Commit

Permalink
Add streaming upload example. (#206)
Browse files Browse the repository at this point in the history
  • Loading branch information
pschrammel authored Jan 3, 2024
1 parent 9170a2a commit 11c4b66
Show file tree
Hide file tree
Showing 5 changed files with 219 additions and 0 deletions.
3 changes: 3 additions & 0 deletions examples/streaming_upload/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
source 'https://rubygems.org'

gem "falcon"
73 changes: 73 additions & 0 deletions examples/streaming_upload/Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
GEM
remote: https://rubygems.org/
specs:
async (2.5.0)
console (~> 1.10)
io-event (~> 1.1)
timers (~> 4.1)
async-container (0.16.12)
async
async-io
async-http (0.60.1)
async (>= 1.25)
async-io (>= 1.28)
async-pool (>= 0.2)
protocol-http (~> 0.24.0)
protocol-http1 (~> 0.15.0)
protocol-http2 (~> 0.15.0)
traces (>= 0.8.0)
async-http-cache (0.4.3)
async-http (~> 0.56)
async-io (1.34.3)
async
async-pool (0.4.0)
async (>= 1.25)
build-environment (1.13.0)
console (1.16.2)
fiber-local
falcon (0.42.3)
async
async-container (~> 0.16.0)
async-http (~> 0.57)
async-http-cache (~> 0.4.0)
async-io (~> 1.22)
build-environment (~> 1.13)
bundler
localhost (~> 1.1)
openssl (~> 3.0)
process-metrics (~> 0.2.0)
protocol-rack (~> 0.1)
samovar (~> 2.1)
fiber-local (1.0.0)
io-event (1.1.7)
localhost (1.1.10)
mapping (1.1.1)
openssl (3.1.0)
process-metrics (0.2.1)
console (~> 1.8)
samovar (~> 2.1)
protocol-hpack (1.4.2)
protocol-http (0.24.1)
protocol-http1 (0.15.0)
protocol-http (~> 0.22)
protocol-http2 (0.15.1)
protocol-hpack (~> 1.4)
protocol-http (~> 0.18)
protocol-rack (0.2.4)
protocol-http (~> 0.23)
rack (>= 1.0)
rack (3.0.7)
samovar (2.1.4)
console (~> 1.0)
mapping (~> 1.0)
timers (4.3.5)
traces (0.9.1)

PLATFORMS
x86_64-linux

DEPENDENCIES
falcon

BUNDLED WITH
2.4.6
66 changes: 66 additions & 0 deletions examples/streaming_upload/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Streaming Upload

# ab Results

The filesize was 82078050 bytes.




## concurrent requests

```
bundle exec falcon host ./falcon.rb
```


```
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:
Server Hostname: localhost
Server Port: 9292

Document Path: /
Document Length: 100 bytes

Concurrency Level: 20
Time taken for tests: 7.873 seconds
Complete requests: 100
Failed requests: 9
(Connect: 0, Receive: 0, Length: 9, Exceptions: 0)
Total transferred: 16280 bytes
Total body sent: 8628660682
HTML transferred: 9989 bytes
Requests per second: 12.70 [#/sec] (mean)
Time per request: 1574.576 [ms] (mean)
Time per request: 78.729 [ms] (mean, across all concurrent requests)
Transfer rate: 2.02 [Kbytes/sec] received
1070310.40 kb/s sent
1070312.42 kb/s total

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 56 44.6 52 141
Processing: 296 1384 725.4 1159 4456
Waiting: 33 126 89.2 121 886
Total: 297 1440 720.8 1236 4464

Percentage of the requests served within a certain time (ms)
50% 1236
66% 1458
75% 1627
80% 1711
90% 2020
95% 3883
98% 4463
99% 4464
100% 4464 (longest request)
```

EOFError errors cause the failed uploads....have to figure out why.
69 changes: 69 additions & 0 deletions examples/streaming_upload/config.ru
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# frozen_string_literal: true

# run with: falcon serve -n 1 -b http://localhost:9292

require 'digest'
require 'securerandom'

module Protocol
module HTTP1
class Connection
def write_continue
@stream.write("HTTP/1.1 100 Continue\r\n\r\n")
@stream.flush
end
end
end
end

class BodyHandler
attr_reader :time, :md5, :size, :uuid

def initialize(input, length)
@uuid = SecureRandom.uuid
@input = input
@length = length
end

def receive
start = Time.now
@md5 = Digest::MD5.new
@size = 0
@done = false

until @done
begin
chunk = @input.read(10_240) # read will raise EOF so we have to check
rescue EOFError
puts "Seems we're done"
chunk = nil
end
if chunk.nil?
@done = true
else
@md5.update(chunk)
@size += chunk.bytesize
@done = true if @length == @size
end
end

@time = Time.now - start
end
end

run lambda { |env|
request = env['protocol.http.request']
handler = BodyHandler.new(env['rack.input'], env['CONTENT_LENGTH'])
puts "#{env['REQUEST_METHOD']} #{handler.uuid}: #{request.path} #{env['CONTENT_LENGTH']}"

if env['REQUEST_METHOD'] == 'POST'
request.connection.write_continue if request.headers['expect'] == ['100-continue']
handler.receive
msg = "Uploaded #{handler.uuid}: #{handler.md5} #{handler.time} #{handler.size}"
puts msg
[200, {}, [msg]]
else
sleep 1
[200, {}, ["#{env['REQUEST_METHOD']}: #{request.path}\n"]]
end
}
8 changes: 8 additions & 0 deletions examples/streaming_upload/falcon.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
load :rack, :supervisor

hostname = File.basename(__dir__)
rack hostname do
endpoint Async::HTTP::Endpoint.parse('http://localhost:9292').with(protocol: Async::HTTP::Protocol::HTTP1)
end

supervisor

0 comments on commit 11c4b66

Please sign in to comment.