ChatGPT

[Anatoly Chernyshev]

Just had an exercise with ChatGPT for Ada.

Here is what my request was: "Produce Ada code for solving a quadratic equation."

And here's the output from ChatGPT (quite flawless forgiving the absence of a math lib in the header):
----------------------------
with Ada.Text_IO; use Ada.Text_IO;

procedure Quadratic_Equation is
A, B, C, D : Float;
X1, X2 : Float;
begin
-- read coefficients A, B, and C from the user
Put("Enter coefficient A: ");
Get(A);

Put("Enter coefficient B: ");
Get(B);

Put("Enter coefficient C: ");
Get(C);

-- calculate the discriminant
D := B*B - 4.0*A*C;

-- check if the discriminant is negative
if D < 0.0 then
Put_Line("No real roots exist.");
else
-- calculate the roots
X1 := (-B + Sqrt(D)) / (2.0*A);
X2 := (-B - Sqrt(D)) / (2.0*A);

-- output the roots
Put_Line("Root 1 = " & Float'Image(X1));
Put_Line("Root 2 = " & Float'Image(X2));
end if;
end Quadratic_Equation;

and an extra remark:

Note that this implementation assumes that the discriminant (B^2 - 4AC) is positive or zero. If the discriminant is negative, then no real roots exist.
-----------------------------------------------

Before crying wolf, my impression is that ChatGPT might be useful for drafting simple pieces of code (say, homeworks). Very soon though one will arrive at a situation when the amount of efforts to explain the task to ChatGPT will be the same as to write the code yourself (I shall call it a limiting Kolmogorov complexity).

What do you think?

[Jerry]

Post by Anatoly Chernyshev
What do you think?

ChatGPT demands my phone number which it will not get. Has anyone found a workaround?
Jerry

[Hou Van Boere]

I have the same concerns. There are websites that allow for temp phone numbers but they never work. Get a burners phone with a pay as you go plan that does not auto-renew. Maybe you already have a retired phone. A simm card will be $10 and a month's worth of talk time perhaps another $10. Wish it was easier :(

[Jeffrey R.Carter]

Post by Anatoly Chernyshev
What do you think?

No doubt there are a large number of such programs in the training data. If it
had simply regurgitated one of those, at least the program would have compiled.
That it couldn't even do as good as that is not impressive.

[Dmitry A. Kazakov]

Post by Jeffrey R.Carter

Post by Anatoly Chernyshev
What do you think?

No doubt there are a large number of such programs in the training data.
If it had simply regurgitated one of those, at least the program would
have compiled. That it couldn't even do as good as that is not impressive.

Right. Fun would be adding qualifiers to the request. E.g. "in extended
precision", "taking arguments from user input" etc. Parroting works up
to some limit.

What I find interesting is that the whole swindle highlights that some
human activities considered creative are in fact not. BTW, it repeats
much earlier dethroning of playing chess as being indicative to human
intellect. The machine easily beats us in chess.

Actually, it is the abilities to sort out garbage and pick up vegetables
which make us intelligent! (:-))

[magardner2010]

Post by Dmitry A. Kazakov

Post by Jeffrey R.Carter

Post by Anatoly Chernyshev
What do you think?

No doubt there are a large number of such programs in the training
data. If it had simply regurgitated one of those, at least the program
would have compiled. That it couldn't even do as good as that is not
impressive.

Right. Fun would be adding qualifiers to the request. E.g. "in extended
precision", "taking arguments from user input" etc. Parroting works up
to some limit.

I have been trying to get chatGPT to help me with a larger Ada project
on and off for the past few months. It not only has no idea regarding
what functions do and do not exist in a given library, but it sometimes
doesn't even understand how private types or tasks work. I will admit,
those are not necessarily super common, but given how easy Ada makes it
to use them, I can honestly say that Ada is the only language in which I
am comfortable writing multithreaded programs that require more
sophistication than a simple fork() call.

So, yeah. ChatGPT knows the syntax of Ada, and it is familiar with
Ada.Text_IO's more common functions. It knows about the existence of
GNAT.Sockets and AdaSockets, but it has no idea on how to use them.

I would be quite confident that that is pretty much it as far as ChatGPT
and Ada goes.

[Anatoly Chernyshev]

Data science people swear it's just a matter of the size of training set used...
I did also a few tests on some simple chemistry problems. ChatGPT looks like a bad but diligent student, who memorized the formulas, but has no clue how to use them. Specifically, units conversions (e.g. between mL, L, m3) is completely off-limits as of now.

[Dmitry A. Kazakov]

Post by Anatoly Chernyshev
Data science people swear it's just a matter of the size of training set used...

They lie. In machine learning overtraining is as much a problem as
undertraining. The simplest example from mathematics is polynomial
interpolation becoming unstable with higher orders.

And this does not even touch contradictory samples requiring retraining
or time constrained samples etc.

Post by Anatoly Chernyshev
I did also a few tests on some simple chemistry problems. ChatGPT looks like a bad but diligent student, who memorized the formulas, but has no clue how to use them. Specifically, units conversions (e.g. between mL, L, m3) is completely off-limits as of now.

One must remember that ChatGPT is nothing but ELIZA on steroids.

https://en.wikipedia.org/wiki/ELIZA

[Ken Burtch]

Post by Dmitry A. Kazakov

Post by Anatoly Chernyshev
Data science people swear it's just a matter of the size of training set used...

They lie. In machine learning overtraining is as much a problem as
undertraining. The simplest example from mathematics is polynomial
interpolation becoming unstable with higher orders.
And this does not even touch contradictory samples requiring retraining
or time constrained samples etc.

Post by Anatoly Chernyshev
I did also a few tests on some simple chemistry problems. ChatGPT looks like a bad but diligent student, who memorized the formulas, but has no clue how to use them. Specifically, units conversions (e.g. between mL, L, m3) is completely off-limits as of now.

One must remember that ChatGPT is nothing but ELIZA on steroids.
https://en.wikipedia.org/wiki/ELIZA
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

For what it's worth on the subject of the chatbot, " "Produce Ada code for solving a quadratic equation." is a terrible choice for a test of ChatGPT as one is asking if it can do a Google search. To test its abilities, you have to pick a challenge that cannot be solved with a Google search.

My short assessment of ChatGPT, with the history of chatbots, are available on my February blog post. I gave it a simple programming problem and it failed 3 times out of 4. It's not surprising as I've learned since February that the chatbot doesn't actually understand programming: it uses examples off the Internet and tries to predict what you might have typed based on keyword patterns. It is an imitation of an imitation, smoke and mirrors. This is why Vint Cerf denounced it. You can read my thoughts on my blog:

https://www.pegasoft.ca/coder/coder_february_2023.html

Ken Burtch

[Martha Simons]

Post by Dmitry A. Kazakov

Post by Anatoly Chernyshev
Data science people swear it's just a matter of the size of training set used...

They lie. In machine learning overtraining is as much a problem as
undertraining. The simplest example from mathematics is polynomial
interpolation becoming unstable with higher orders.
And this does not even touch contradictory samples requiring retraining
or time constrained samples etc.

Post by Anatoly Chernyshev
I did also a few tests on some simple chemistry problems. ChatGPT looks like a bad but diligent student, who memorized the formulas, but has no clue how to use them. Specifically, units conversions (e.g. between mL, L, m3) is completely off-limits as of now.

One must remember that ChatGPT is nothing but ELIZA on steroids.
https://en.wikipedia.org/wiki/ELIZA
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

For what it's worth on the subject of the chatbot, " "Produce Ada code for solving a quadratic equation." is a terrible choice for a test of ChatGPT as one is asking if it can do a Google search. To test its abilities, you have to pick a challenge that cannot be solved with a Google search.
https://www.pegasoft.ca/coder/coder_february_2023.html
Ken Burtch

https://chat.openai.com/g/g-2IqDEDp7c-essay-typer
https://chat.openai.com/g/g-6sIiosQhh-write-my-essay-bot
https://chat.openai.com/g/g-ldMROHWp6-gpt-paraphrasing-tool
https://chat.openai.com/g/g-2IqDEDp7c-essay-typer
https://chat.openai.com/g/g-wznfN7mwL-chatgpt-essay-writer
https://chat.openai.com/g/g-BKo6liIEe-ai-essay-writer-by-eduwriter-ai

https://www.bulbapp.com/u/ai-essay-writer
https://www.bulbapp.com/u/essay-typer

[Nioclásán Caileán de Ghlostéir]

Post by Dmitry A. Kazakov

Post by Anatoly Chernyshev
Data science people swear it's just a matter of the size of training set used...

They lie. In machine learning overtraining is as much a problem as
undertraining. The simplest example from mathematics is polynomial
interpolation becoming unstable with higher orders.
And this does not even touch contradictory samples requiring retraining
or time constrained samples etc.

Post by Anatoly Chernyshev
I did also a few tests on some simple chemistry problems. ChatGPT looks like a bad but diligent student, who memorized the formulas, but has no clue how to use them. Specifically, units conversions (e.g. between mL, L, m3) is completely off-limits as of now.

One must remember that ChatGPT is nothing but ELIZA on steroids.
https://en.wikipedia.org/wiki/ELIZA
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

For what it's worth on the subject of the chatbot, " "Produce Ada code for solving a quadratic equation." is a terrible choice for a test of ChatGPT as one is asking if it can do a Google search. To test its abilities, you have to pick a challenge that cannot be solved with a Google search.

My short assessment of ChatGPT, with the history of chatbots, are available on my February blog post. I gave it a simple programming problem and it failed 3 times out of 4. It's not surprising as I've learned since February that the chatbot doesn't actually understand programming: it uses examples off the Internet and tries to predict what you might have typed based on keyword patterns. It is an imitation of an imitation, smoke and mirrors. This is why Vint Cerf denounced it. You can read my thoughts on my blog:

https://www.pegasoft.ca/coder/coder_february_2023.html

Ken Burtch"


As an experiment I asked WWW.Perplexity.AI "Why is ChatGPT bad at Ada?" so
Perplexity alleged:
"Answer
ChatGPT's performance with the Ada programming language can be perceived
as inadequate due to several factors related to both the language's
characteristics and the limitations of the AI model itself.
Factors Contributing to ChatGPT's Performance Issues with Ada
1. Complexity of Ada:
Ada is known for its strong emphasis on safety and correctness, which
includes features like runtime checks and strong typing. While these
features are beneficial for developing reliable software, they can also
make the language more complex and verbose. This complexity may lead to
misunderstandings or oversights when ChatGPT attempts to generate or
analyze Ada code, especially for more intricate tasks12.
2. Performance Overhead:
The safety features inherent in Ada, such as range checking and bounds
checking, introduce additional overhead that can affect performance. This
overhead is crucial for safety-critical applications but may not be
well-handled by ChatGPT when generating code or discussing performance
optimizations. The model may struggle to provide accurate assessments or
optimizations for Ada code compared to languages like C or C++, which are
often perceived as more straightforward in terms of low-level
optimizations1.
3. Limitations of AI Understanding:
ChatGPT operates based on patterns in the data it has been trained on. As
a result, it may not fully grasp the nuances of niche programming
languages like Ada, particularly when users ask complex questions or
request sophisticated code examples. This limitation can lead to
incorrect or overly simplistic responses when users seek detailed help
with Ada programming23.
4. User Expectations and Experience:
Users have reported that while ChatGPT can handle basic queries related
to Ada, it often fails with more complex tasks. This inconsistency can
lead to frustration, as users expect a level of expertise that the model
may not be able to provide due to its training limitations and the
inherent complexities of the Ada language25.
5. Hallucination and Inaccuracy:
ChatGPT has a tendency to "hallucinate" or generate incorrect
information, particularly in specialized fields where it lacks extensive
training data. This issue is exacerbated in niche programming languages
like Ada, where accurate information may not be as prevalent in its
training corpus. Consequently, users might receive misleading advice or
code snippets that do not function as intended34.
In summary, ChatGPT's challenges with Ada stem from the language's
complexity, performance overhead from safety features, limitations in AI
understanding, user expectations, and a propensity for inaccuracies.
These factors collectively contribute to a perception that ChatGPT is not
well-suited for generating or analyzing Ada code effectively."