Skip to content

Commit

Permalink
polish
Browse files Browse the repository at this point in the history
  • Loading branch information
konfig-bot committed Oct 25, 2023
1 parent 71ce8c3 commit 83977dc
Showing 1 changed file with 50 additions and 40 deletions.
90 changes: 50 additions & 40 deletions generator/konfig-docs/blog/2023/10/23-github-copilot.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ import Head from '@docusaurus/Head';
<meta property="og:title" content="I Reviewed 1,000s of Opinions on Github Copilot"/>
</Head>

GitHub Copilot has recently taken the software engineering world by storm and
just hit a milestone of $100M ARR, which is enough to stand alone as a publicly
listed company. Funding is also pouring into code-focused LLM use cases.
GitHub Copilot has recently taken the software engineering world by storm,
hitting a milestone of $100M ARR. This achievement alone qualifies it to be a
publicly listed company. Meanwhile, funding continues to flow into code-focused
LLM use cases.

<Carousel.Wrapper>
<Carousel.Slide>
Expand Down Expand Up @@ -66,7 +67,7 @@ only thought-provoking opinions.
</Figure>

Next, I transcribed these discussions onto a whiteboard, organizing them into
"Anti-Copilot" (👍), "Pro-Copilot" (👎), or "Neutral" (🧐) categories, and then clustering them
"Anti-Copilot" (👎), "Pro-Copilot" (👍), or "Neutral" (🧐) categories, and then clustering them
into distinct opinions. Each section in this post showcases an opinion while
referencing pertinent discussions.

Expand All @@ -76,9 +77,10 @@ referencing pertinent discussions.

## 👎 Copilot produces bad results

LLMs are probabilistic models, which means they are not always correct. This is
especially true for Copilot, which is trained on a corpus of code that may not
be representative of the code that a developer writes. As a result, Copilot can
LLMs operate as probabilistic models, implying they aren't always correct. This
is especially true for Copilot, which is trained on a corpus of code that may
not be representative of the code that a developer writes. As a result, Copilot
can
produce consistently bad results.

<Carousel.Wrapper>
Expand Down Expand Up @@ -136,20 +138,23 @@ produce consistently bad results.

#### Key Takeaway 🔑

Developers expect their tools to be reliable.
Developers expect reliability from their tools.

Copilot is not reliable, and therefore, certain developers have a tough time
wrestling with its output. Copilot would sometimes lie or produce bad results
for a vast majority of the time. For people who worry about job security, fear
nothing, because Copilot is not going to replace you anytime soon.
wrestling with its output. Copilot lies or produces bad results for a vast
majority of the time. This can be exceedingly frustrating for developers who are
expecting Copilot to deliver on its promise of writing code for you. After some
bad experiences, some developers have even stopped using Copilot altogether.


For people who worry about job security, fear nothing, because Copilot is not
going to replace you anytime soon.


## 👎 Copilot creates more problems than solutions

Copilot is a tool that is supposed to help developers write code. However, it
can also create more problems than solutions. Code requires 100% accuracy, and
inaccuracy can lead you down a rabbit hole of debugging. Often wasting time or flat out
breaking your code.
Copilot is a tool that is supposed to help developers write code. However, its
unreliable results creates more problems than solutions.

<Carousel.Wrapper>
<Carousel.Slide>
Expand Down Expand Up @@ -186,14 +191,15 @@ breaking your code.

#### Key Takeaway 🔑

Copilot can mess up your code.
Copilot can waste your time.

In some cases, this is frustrating enough for developers to stop using Copilot
altogether. Just like managing a junior developer, Copilot requires a lot of
oversight. Sometimes subtle bugs can take more time to debug and fix than
writing the code yourself. For some developers who find the output to be too
inaccurate, Copilot becomes an interference and ultimately doesn't save them any
time.
Code requires 100% accuracy, and inaccuracy can lead you down a rabbit hole of
debugging. Often wasting time or flat out breaking your code. In some cases,
this is frustrating enough for developers to stop using Copilot altogether.
Just like managing a junior developer, Copilot requires a lot of oversight.
Sometimes subtle bugs can take more time to debug and fix than writing the code
yourself. For some developers who find the output to be too inaccurate, Copilot
becomes an interference and ultimately doesn't save them any time.

## 👍 Copilot helps you write software faster

Expand Down Expand Up @@ -252,13 +258,17 @@ take care of the boring stuff, it can be a powerful tool.

Copilot increases productivity.

Often time developers are faced with mundane and repetitive tasks. Given enough
context, Copilot can do these tasks for you with sufficient accuracy. For some
Often, developers face mundane and repetitive tasks. Given enough context,
Copilot can do these tasks for you with sufficient accuracy. For some
developers, these tasks can be a significant time sink, and Copilot can help you
get that time back. A 10-20% increase in productivity is nothing to scoff at.
Assuming an engineer is paid $100k/yr and they are 5% more productive. At the
yearly price of [$100/yr](https://github.com/features/copilot#pricing), you are
creating $4900 of value for your company.
get that time back.

Based on the mentioned 10-20% increase in productivity, such an improvement is
substantial. For the sake of a conservative analysis, let's consider the lower
bound: if we assume an engineer is paid $100k/yr and becomes just 5% more
productive (half of the 10% reference), then with a yearly cost of
[$100](https://github.com/features/copilot#pricing) for Copilot, the tool brings
in an added value of $4900 for the company.

## 👍 Copilot helps you write better software

Expand Down Expand Up @@ -303,10 +313,11 @@ and happier developers.

Copilot has enough intelligence to help you write better software.

This is especially true for junior developers who are still learning the ropes.
It can drastically make mundane tasks like documentation and testing easier,
giving developers more time to focus on the bigger picture while maintaining a
high standard of code quality.
This holds especially true for junior developers still learning the ropes. It
can drastically make mundane tasks like documentation and testing easier, giving
developers more time to focus on the bigger picture while maintaining a high
standard of code quality. Multiplying this effect across an engineering team
leads to a higher quality codebase—the ultimate dream for engineering leaders.

## 🧐 Copilot is like a calculator

Expand Down Expand Up @@ -349,24 +360,23 @@ you still need to know how to write code.

#### Key Takeaway 🔑

Similar to how calculators allow mathematicians to solve problems faster,
Copilot keeps developers from worrying about repetitive tasks.
Just as calculators enable mathematicians to solve problems more quickly,
Copilot spares developers from focusing on repetitive tasks.

However, just like calculators, Copilot does not help you make sense of the
problem. You still need to know how to solve problems, and you still need to
know how to write code.

# Conclusion

Depending on who you ask, Copilot is either a blessing or a curse. Some
developers love it, while others hate it. For those who praise it,
Copilot is a tool that can help you write code faster and better. For those
who hate it, Copilot is a tool that can create more problems than solutions.
Opinions on Copilot vary; some see it as a blessing, while others regard it as a
curse. For its proponents, Copilot is a valuable tool that enhances coding speed
and quality. However, critics argue it introduces more issues than it resolves.

I suspect that the complexity of a task makes a big difference in the quality of
output. Working on tasks that require more context will inevitably lead to worse
results. But if you just treat Copilot as a tool that can help take care of the
boring stuff, it can be a powerful tool.
results. Yet, when viewed simply as a tool to handle the mundane aspects of
coding, Copilot reveals its potential.

<Admonition type="info" title="Our opinion">
At [Konfig](https://konfigthis.com), we use Copilot and ChatGPT daily and have
Expand Down

0 comments on commit 83977dc

Please sign in to comment.