Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggested changes #1752

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
2a00258
Add suggestion to comment for supported violations
salbertson Nov 8, 2019
f05afa1
Build violation with source
salbertson Nov 8, 2019
9d6dd92
Make suggestions for missing space after comma
salbertson Nov 8, 2019
8b5f0cf
Add source to violations
salbertson Nov 12, 2019
735010b
Only make one suggestion
salbertson Nov 16, 2019
328f947
Use more realistic source examples in tests
salbertson Nov 16, 2019
e24fe6b
Encrypt the single lines of source code
salbertson Nov 16, 2019
8a1b92c
Style updates
salbertson Nov 16, 2019
2a58fe8
More style changes
salbertson Nov 16, 2019
cdd2f6f
Clean up decryption logic a bit
salbertson Nov 26, 2019
7828c12
Remove concurrent migration option
salbertson Nov 26, 2019
774d389
Make suggested changes to security doc
salbertson Nov 26, 2019
47b369d
Drop unnecessary variable
salbertson Nov 26, 2019
824ce4a
Use if instead of unless
salbertson Nov 26, 2019
b8ddf22
Revert most changes to decrypt
salbertson Nov 26, 2019
2c40d3e
Add test coverage and simplify #source
salbertson Mar 26, 2020
3e7f14b
Use consistent decrypt coding style
salbertson Mar 26, 2020
79f5ad2
Update tests to include source when required
salbertson Mar 26, 2020
1ab171b
Add trailing comma
salbertson Mar 26, 2020
143fe95
Upgrade administrate
salbertson Mar 26, 2020
69aa469
Upgrade rails
salbertson Mar 26, 2020
f1f9b18
Upgrade puma
salbertson Mar 26, 2020
7d1d863
Remove messages instance variable
salbertson Mar 27, 2020
405902c
Don't update violation.source
salbertson Mar 27, 2020
27f595a
Add newline to suggestions
salbertson Mar 30, 2020
d48f119
Add newlines to tests to match new suggestion format
salbertson Mar 30, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ source "https://rubygems.org"
ruby "2.6.5"

gem "active_model_serializers", "0.10.10"
gem "administrate", "0.12.0"
gem "administrate", "0.13.0"
gem "analytics-ruby", "~> 2.2.2", require: "segment/analytics"
gem "attr_extras"
gem "autoprefixer-rails"
Expand Down
114 changes: 57 additions & 57 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,38 +1,38 @@
GEM
remote: https://rubygems.org/
specs:
actioncable (6.0.2.1)
actionpack (= 6.0.2.1)
actioncable (6.0.2.2)
actionpack (= 6.0.2.2)
nio4r (~> 2.0)
websocket-driver (>= 0.6.1)
actionmailbox (6.0.2.1)
actionpack (= 6.0.2.1)
activejob (= 6.0.2.1)
activerecord (= 6.0.2.1)
activestorage (= 6.0.2.1)
activesupport (= 6.0.2.1)
actionmailbox (6.0.2.2)
actionpack (= 6.0.2.2)
activejob (= 6.0.2.2)
activerecord (= 6.0.2.2)
activestorage (= 6.0.2.2)
activesupport (= 6.0.2.2)
mail (>= 2.7.1)
actionmailer (6.0.2.1)
actionpack (= 6.0.2.1)
actionview (= 6.0.2.1)
activejob (= 6.0.2.1)
actionmailer (6.0.2.2)
actionpack (= 6.0.2.2)
actionview (= 6.0.2.2)
activejob (= 6.0.2.2)
mail (~> 2.5, >= 2.5.4)
rails-dom-testing (~> 2.0)
actionpack (6.0.2.1)
actionview (= 6.0.2.1)
activesupport (= 6.0.2.1)
actionpack (6.0.2.2)
actionview (= 6.0.2.2)
activesupport (= 6.0.2.2)
rack (~> 2.0, >= 2.0.8)
rack-test (>= 0.6.3)
rails-dom-testing (~> 2.0)
rails-html-sanitizer (~> 1.0, >= 1.2.0)
actiontext (6.0.2.1)
actionpack (= 6.0.2.1)
activerecord (= 6.0.2.1)
activestorage (= 6.0.2.1)
activesupport (= 6.0.2.1)
actiontext (6.0.2.2)
actionpack (= 6.0.2.2)
activerecord (= 6.0.2.2)
activestorage (= 6.0.2.2)
activesupport (= 6.0.2.2)
nokogiri (>= 1.8.5)
actionview (6.0.2.1)
activesupport (= 6.0.2.1)
actionview (6.0.2.2)
activesupport (= 6.0.2.2)
builder (~> 3.1)
erubi (~> 1.4)
rails-dom-testing (~> 2.0)
Expand All @@ -42,28 +42,28 @@ GEM
activemodel (>= 4.1, < 6.1)
case_transform (>= 0.2)
jsonapi-renderer (>= 0.1.1.beta1, < 0.3)
activejob (6.0.2.1)
activesupport (= 6.0.2.1)
activejob (6.0.2.2)
activesupport (= 6.0.2.2)
globalid (>= 0.3.6)
activemodel (6.0.2.1)
activesupport (= 6.0.2.1)
activerecord (6.0.2.1)
activemodel (= 6.0.2.1)
activesupport (= 6.0.2.1)
activestorage (6.0.2.1)
actionpack (= 6.0.2.1)
activejob (= 6.0.2.1)
activerecord (= 6.0.2.1)
activemodel (6.0.2.2)
activesupport (= 6.0.2.2)
activerecord (6.0.2.2)
activemodel (= 6.0.2.2)
activesupport (= 6.0.2.2)
activestorage (6.0.2.2)
actionpack (= 6.0.2.2)
activejob (= 6.0.2.2)
activerecord (= 6.0.2.2)
marcel (~> 0.3.1)
activesupport (6.0.2.1)
activesupport (6.0.2.2)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
tzinfo (~> 1.1)
zeitwerk (~> 2.2)
addressable (2.7.0)
public_suffix (>= 2.0.2, < 5.0)
administrate (0.12.0)
administrate (0.13.0)
actionpack (>= 4.2)
actionview (>= 4.2)
activerecord (>= 4.2)
Expand All @@ -76,7 +76,7 @@ GEM
selectize-rails (~> 0.6)
analytics-ruby (2.2.8)
attr_extras (6.2.3)
autoprefixer-rails (9.7.4)
autoprefixer-rails (9.7.5)
execjs
bourbon (5.1.0)
sass (~> 3.4)
Expand Down Expand Up @@ -175,7 +175,7 @@ GEM
mini_mime (>= 0.1.1)
marcel (0.3.3)
mimemagic (~> 0.3.2)
method_source (0.9.2)
method_source (1.0.0)
mimemagic (0.3.4)
mini_mime (1.0.2)
mini_portile2 (2.4.0)
Expand All @@ -191,7 +191,7 @@ GEM
sass (>= 3.3)
thor (~> 0.19)
nio4r (2.5.2)
nokogiri (1.10.8)
nokogiri (1.10.9)
mini_portile2 (~> 2.4.0)
normalize-rails (4.1.1)
oauth2 (1.4.4)
Expand Down Expand Up @@ -220,7 +220,7 @@ GEM
pathspec (0.2.1)
pg (1.2.2)
public_suffix (4.0.3)
puma (4.3.1)
puma (4.3.3)
nio4r (~> 2.0)
rack (2.2.2)
rack-protection (2.0.8.1)
Expand All @@ -230,20 +230,20 @@ GEM
rack-test (1.1.0)
rack (>= 1.0, < 3)
rack-timeout (0.6.0)
rails (6.0.2.1)
actioncable (= 6.0.2.1)
actionmailbox (= 6.0.2.1)
actionmailer (= 6.0.2.1)
actionpack (= 6.0.2.1)
actiontext (= 6.0.2.1)
actionview (= 6.0.2.1)
activejob (= 6.0.2.1)
activemodel (= 6.0.2.1)
activerecord (= 6.0.2.1)
activestorage (= 6.0.2.1)
activesupport (= 6.0.2.1)
rails (6.0.2.2)
actioncable (= 6.0.2.2)
actionmailbox (= 6.0.2.2)
actionmailer (= 6.0.2.2)
actionpack (= 6.0.2.2)
actiontext (= 6.0.2.2)
actionview (= 6.0.2.2)
activejob (= 6.0.2.2)
activemodel (= 6.0.2.2)
activerecord (= 6.0.2.2)
activestorage (= 6.0.2.2)
activesupport (= 6.0.2.2)
bundler (>= 1.3.0)
railties (= 6.0.2.1)
railties (= 6.0.2.2)
sprockets-rails (>= 2.0.0)
rails-dom-testing (2.0.3)
activesupport (>= 4.2.0)
Expand All @@ -255,9 +255,9 @@ GEM
rails_stdout_logging
rails_serve_static_assets (0.0.5)
rails_stdout_logging (0.0.5)
railties (6.0.2.1)
actionpack (= 6.0.2.1)
activesupport (= 6.0.2.1)
railties (6.0.2.2)
actionpack (= 6.0.2.2)
activesupport (= 6.0.2.2)
method_source
rake (>= 0.8.7)
thor (>= 0.20.3, < 2.0)
Expand Down Expand Up @@ -364,14 +364,14 @@ GEM
websocket-extensions (0.1.4)
xpath (3.2.0)
nokogiri (~> 1.8)
zeitwerk (2.2.2)
zeitwerk (2.3.0)

PLATFORMS
ruby

DEPENDENCIES
active_model_serializers (= 0.10.10)
administrate (= 0.12.0)
administrate (= 0.13.0)
analytics-ruby (~> 2.2.2)
attr_extras
autoprefixer-rails
Expand Down
3 changes: 2 additions & 1 deletion app/models/file_review.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@ class FileReview < ApplicationRecord
belongs_to :build
has_many :violations, dependent: :destroy

def build_violation(line, message)
def build_violation(line, message, source)
if line.changed?
violation = find_or_build_violation(line)
violation.add_message(message)
violation.patch_position = line.patch_position
violation.source = source
end
end

Expand Down
8 changes: 2 additions & 6 deletions app/models/user.rb
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,11 @@ def has_active_repos?
end

def token=(value)
encrypted_token = crypt.encrypt_and_sign(value)
write_attribute(:token, encrypted_token)
self[:token] = crypt.encrypt_and_sign(value)
end

def token
encrypted_token = read_attribute(:token)
unless encrypted_token.nil?
crypt.decrypt_and_verify(encrypted_token)
end
crypt.decrypt_and_verify(self[:token]) if self[:token]
gylaz marked this conversation as resolved.
Show resolved Hide resolved
end

def payment_gateway_subscription
Expand Down
15 changes: 15 additions & 0 deletions app/models/violation.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,19 @@ def add_message(message)
def messages
self[:messages].uniq
end

def source=(value)
self[:source] = crypt.encrypt_and_sign(value)
end

def source
crypt.decrypt_and_verify(self[:source]) if self[:source]
end

private

def crypt
secret_key_base = Rails.application.secrets.secret_key_base
ActiveSupport::MessageEncryptor.new(secret_key_base[0, 32], secret_key_base)
end
end
7 changes: 6 additions & 1 deletion app/services/complete_file_review.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,12 @@ def complete_file_review
def build_file_review_violations
attributes.fetch(:violations).each do |violation|
line = commit_file.line_at(violation.fetch(:line))
file_review.build_violation(line, violation.fetch(:message))

file_review.build_violation(
line,
violation.fetch(:message),
violation.fetch(:source),
)
end
end

Expand Down
2 changes: 1 addition & 1 deletion app/services/submit_review.rb
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def build_comment(violation)
{
path: violation.filename,
position: violation.patch_position,
body: violation.messages.join(CommentingPolicy::COMMENT_LINE_DELIMITER),
body: SuggestChanges.call(violation),
}
end

Expand Down
41 changes: 41 additions & 0 deletions app/services/suggest_changes.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
class SuggestChanges
static_facade :call

def initialize(violation)
@violation = violation
@suggestion = ""
end

def call
violation.messages.each do |message|
gylaz marked this conversation as resolved.
Show resolved Hide resolved
apply_suggestion(message)
break if suggestion.present?
end
gylaz marked this conversation as resolved.
Show resolved Hide resolved

messages = violation.messages.join(CommentingPolicy::COMMENT_LINE_DELIMITER)

if suggestion.blank?
messages
else
messages + "<br>\n```suggestion\n#{suggestion}\n```"
end
end

private

attr_reader :violation
attr_accessor :suggestion

def apply_suggestion(message)
case message
when /Trailing whitespace detected/
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gylaz we need to figure out a better way to generate these suggestions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, though not sure how "smart" we want to get about these.

self.suggestion = violation.source.gsub(/(.*)\s$/, '\1')
when /A space is required after ','/
self.suggestion = violation.source.gsub(/(,)([^ ])/, ', \2')
gylaz marked this conversation as resolved.
Show resolved Hide resolved
when /Put a comma after the last parameter of a multiline method call/
self.suggestion = violation.source + ","
when /Missing semicolon/
self.suggestion = violation.source + ";"
end
end
end
5 changes: 5 additions & 0 deletions db/migrate/20191108222026_add_source_to_violations.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class AddSourceToViolations < ActiveRecord::Migration[5.1]
def change
add_column :violations, :source, :string
end
end
1 change: 1 addition & 0 deletions db/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@
t.integer "line_number"
t.text "messages", default: [], null: false, array: true
t.integer "file_review_id", null: false
t.string "source"
t.index ["file_review_id"], name: "index_violations_on_file_review_id"
end

Expand Down
8 changes: 5 additions & 3 deletions doc/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,9 +218,11 @@ to comment about the violations on the pull request.

`CompleteFileReview` saves each violation, the line number, and patch position,
in the `violations` table of our Postgres database.
We do not save any of your code in Postgres.
We do store the contents of the files to review temporarily in Sidekiq, and it
gets cleared out as soon as the job is processed.
We also store single lines of code that are used to report violations and
suggested changes. The data is encrypted and encoded similar to tokens using
`ActiveSupport::MessageEncryptor`. We also store the contents of the files to
review temporarily in Redis, but that is removed as soon as the Sidekiq job is
processed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


To browse the portions of the codebase related to
receiving and processing pull request notifications,
Expand Down
Loading