Chat GPt - a way to go for reliable information

[Stuart France]

Nevertheless, what happened happened. I was using their "Try GPT" service. I'm not registered with an account linked to my email address for the obvious reason that I can imagine that it would potentially enable them to build a picture, record a history, sell it or circulate it.

Yesterday I also had a Zoom with a real person with a PhD in AI. Subject: the analysis of visitor data. After an hour we agreed that AI has nothing to offer over and above what can be done with traditional statistical methods. For example, we have a run of 100 days of visitor counts from some site. Ask AI "what do you make of that?" This kind of open question implies actual thinking, actual understanding. Would AI think of calculating median, mean, standard deviation, then do it, then start looking for "something". As a starting point, are the daily numbers credible? The Tower of London will be in a different footfall league to OFD. What if the mean and median are about the same, or not? What does that tell you? Suppose one particular date had a count which was three SDs above the mean? Is it reasonable to expect a few really wacky results within a run of 100 due to pure chance and natural variability? Suppose that date was a Saturday, which means seeing the need and bothering to check the weekday this date corresponds to? Suppose we have hourly totals as well as daily ones. Suppose all the excess activity occurred between 10am-3pm, or alternatively if it occurred between 2am-6am, then what would that suggest? Did it happen in June or December? Do we have comparative data for any other paths nearby or a car park, and were they affected or not on that date? Do we know anything about the weather then? And so forth.

Interpretation requires understanding wider contexts, deep knowledge of how the world operates, and in particular the human world, in search of plausible explanations of some value or interest to a client. These are far away from the kind of questions like "[what is a] cave" or "how do you tie a bowline knot" which are basically look-ups on encyclopaedic websites like Wikipedia or what Google does.

 

[snebbit]

Agree with Ari that it sounds nothing like actual GPT replies, it would never just say 'no results'. It's likely to have plenty of fake clones pulling traffic in for advertising/account harvesting. Check you're on openai.com (or chatgptonline.ai if you don't want to create an account)

It's true, it's not a stand-in for a human mind right now. But it's still very impressive as a milestone in the AI/ML journey we're all on whether we like it or not. From my point of view it saves me hours of work every single day writing boring and repetitive code, leaving me with more time to bring its pieces together and work on the bigger architecture. It's like having a free, instant, tireless junior developer.

That said, I asked it for its favourite Leck Fell through-trips the other day, it came up with gems such as Wretched Rabbit to Jingling Pot, 'a classic Yorkshire Dales caving trip'. After letting it know that Wretched Rabbit is currently closed (among other issues with that trip), it apologised and suggested its next favourite, Pippikin to Long Churn. So it's definitely bringing enough bullshit to be considered a true caver

 

[snebbit]

@Stuart France I had a chat with ChatGPT based on your example, pasting in a random dataset of dates and visitor counts between 1 and 10. Here's what it made of that

Here is some data for daily visitor counts at a site. Provide some basic statistics and any interpretations you might have:
To provide some basic statistics, we can calculate the mean, median, mode, and standard deviation of the visitor counts: Mean: 4.96 Median: 4 Mode: 3 Standard Deviation: 3.09 The mean visitor count is about 4.96, which indicates that the site receives approximately 5 visitors per day on average. The median visitor count is 4, which is less than the mean, suggesting that the data is slightly skewed towards higher values. The mode of the data is 3, indicating that this is the most frequent visitor count in the dataset. The standard deviation of 3.09 is relatively high, indicating that the data is spread out over a range of values. Based on these statistics, we can infer that the site's daily visitor count varies significantly from day to day, with some days having no visitors and others having as many as 10. It's unclear from this dataset whether there are any patterns or trends in the visitor count over time. However, we can plot the visitor counts over time to see if there are any trends or patterns. It's also possible that there could be seasonal or monthly trends that affect the visitor count. Additional data, such as the type of website, the location, or promotional events, could also help to explain the variability in the visitor counts.

Is there any trend based on month?
To determine whether there is a trend based on month, we can group the visitor counts by month and calculate the mean visitor count for each month. Here are the monthly means based on the data provided: January: 5 February: 4 March: 5 April: 2 May: N/A June: 4.66 July: 3 August: 4.2 September: 4.33 October: 8.33 November: 6.6 December: 4.33 From this analysis, we can see that October and November have the highest average visitor counts, with 8.33 and 6.6 visitors per day, respectively. April has the lowest average visitor count, with only 2 visitors per day. However, it's important to note that this dataset only includes data from one year, so it's difficult to draw any definitive conclusions about monthly trends. Additionally, the data is limited, and there could be other factors besides month that influence the visitor count, such as holidays, events, or seasonal changes. Therefore, it would be helpful to collect more data over a longer period of time to confirm whether there is a trend based on month.

 

[Stuart France]

The problem here is that, generally speaking, mid-July to end-August is by far the busiest time of the year for a footpath at least, for the obvious reasons, obvious to westernized humans anyway. Furthermore, weekends will always be busier than midweek except at summer holiday destinations where people tend to stay in the area for 1-2 weeks which flattens the day-of-week profile. On the other hand, if the data is for a town centre park then Sunday will be the quietest day of the week by a big margin.

The monthly totals will rise towards July, but August won't have the highest footfall, again for the obvious reason that people in most towns go away on holiday: so September will be busier than August when they come back home, really, then it falls month by month to December and Christmas does not rescue the street footfall situation. If the town is a holiday destination then August will be its busiest month. There isn't an annual trend by month, monthly totals are driven by other factors that come and go, but clearly ChatGPT is 'imagining' a simple monthly trend, presumbly one going straight uphill all the way from Jan to Dec, or vice versa, but the real world doesn't work like that, the human world anyway: nowhere on Earth does footfall go straight up or straight down from Jan to Dec, a yoyo, unless you know better.

It's all very well to casually mention holidays, weather, etc in the system's response, but that is just common sense that could be pulled from any reports on footfall patterns, particularly ones related to the outdoors leisure. The point is that ChatGPT has not recognized the lack of correlation with common sense or seen fit to ask about the nature of the example site (town, country, coast etc) because it hasn't got any common sense. If I saw August as being the quietest month of the year and November being twice August's level in a place like Pembrokeshire or the Yorks Dales or even Queen Street Cardiff, then I'd be flagging that dataset as absurd.

Here's some real world round numbers for 2019 (the last pre-covid year) for you to think about for a large set of National Nature Reserves by month: Jan 84K, Feb 100K, Mar 102K, Apr 176K, May 166K, Jun 160K, Jul 209K, Aug 256K, Sep 138K, Oct 107K, Nov 70K, Dec 78K. Total 1.6M, and by the way Easter was late. In the real world if you've been at all awake for the past 2 years or so in the recent history of humankind then there's this massive thing called Covid affecting footfall in all manner of ways - which ChatGPT has overlooked!

So, on ability to churn out published material and process a given set of numbers presented in a natural language wrapper, it's promising, but where are the links to common sense and the ever-broading experience of living a life that are the very foundation of real intelligence.

 

[mikem]

But then, how much comprehension would you expect from any other 4.5 month old... (or you can consider it an 8 year old that has just been released from home schooling!)

For national nature reserves, a higher percentage of winter figures will be same people visiting a local site & Nov/Dec are lowest as they will already have seen most of the species available that year, whilst January is start of new year list.

 

[Stuart France]

Well we don't have evidence of repeat NNR visitors because our sensor is not facial recognition or doing mobile phone sniffing, not that all of these places have MNO coverage nor do all visitors carry mobile phones into such venues. I don't.

When I was a postgrad, my uni paid me (not handsomely but useful contributions to my welfare) to mark the computer science undergrads programming assignments. It was a revelation into the way human minds can work and the products of teamwork. This was the 1970s and social media then was the coffee bar and the local pizzeria.

Suppose you asked the undergrads to code an algorithm to output the ISO Standard Week Number for a given date. This is severe test of competence as it has taken even Microsoft over a decade to get this totally right in Windows. I gave up on them few years back and coded my own week number algorithm. No internet back in those days, so the students had to invent some algorithm for themselves, either individually or pool their talents in the coffee bar, in which case all their homework looked suspiciously alike other than for their trivial global edits to change variable names or alter the page layout intended to fool the assessor (me) into believing every submission was original. Of course many of these student algorithms were flawed, and the very same flaws kept cropping up, as you'd expect from group efforts over coffee.

Fast forward to now. ChatGPT is the same as the above but without the coffee bar part which has become scraping the internet for material of unknown accuracy written by unknown people. Generative answers can simply be garbage-in-garbage-out eloquently regurgitated as creative. The trouble is many ordinary folk are taken in by computer-said-so-it-must-be-right especially when presented in natural language which bypasses current humankind's critical faculties (if possessed at all) by giving the false impression that the prosaic output was written in all honesty by a real sentient, well-meaning human being when it wasn't.

Until quite recently the buzz word was "algorithm". People had no idea what one was, but it sounded great. All you had to do was plaster "algorithm" and ideally "patent" all over your website and and your share price and trading volumes rocketed overnight. Before that it was "block chain". Yes, lots of money will be made and lost with "Generative AI" as with "Crypto" before the light dawns.

ChatGPT's offerings have no place in medical, air traffic control, nuclear bomb firmware, my pension, or other 'safety' critical applications. Just don't go there. I would not employ anyone in our little business who downloads and uses code of unknown origin and takes reliability on trust.

 

[mikem]

But most of the b*llsh*t that's already online was posted & edited by people...

 

[Stuart France]

Correct. Published material is 'scraped', rebuilt into databases, transformed, and then fed back to users by Generative AI as if it is original and reliable when surely it isn't? That's the point I'm making. Unless you have source material that is either infallible or trustworthy, which demonstrates creativity, insight and manifests intelligence like the ability to "reason by analogy" on the basis of one's life experience, as intelligent real people often do, then all you have is attractively presented regurgitation of what's been published already regardless of accuracy and bias.

Attractively presented regurgitation of unvetted material is not intelligence, it's just a database operation.

 

[aricooperdavis]

Attractively presented regurgitation of unvetted material is not intelligence, it's just a database operation.

Absolutely, but it would be foolish to write it off as valueless. I expect it to be heavily involved in compiling many pub quizzes, homework problems, and advertising copy in the very near future, if not already. And none of those will be novel or groundbreaking, but when are they anyway?

 

[JasonC]

A pub quiz based solely on unchecked ChatGPT responses would, I think, lead to some "spirited discussions" in the pub

 

[wellyjen]

Correct. Published material is 'scraped', rebuilt into databases, transformed, and then fed back to users by Generative AI as if it is original and reliable when surely it isn't? That's the point I'm making. Unless you have source material that is either infallible or trustworthy, which demonstrates creativity, insight and manifests intelligence like the ability to "reason by analogy" on the basis of one's life experience, as intelligent real people often do, then all you have is attractively presented regurgitation of what's been published already regardless of accuracy and bias.

Attractively presented regurgitation of unvetted material is not intelligence, it's just a database operation.

And when the next generation or two of these AI systems includes an increasing proportion of AI generated content scraped from the internet as well as human generated material, then it will likely degenerate in to deep weirdness.

 

[pwhole]

You only have to start doing stuff like cave and mine research to realise just how much material still isn't on the internet

 

[StealthYak]

I asked to tell me what it knew about the MNRC, and the answer was convincing but wrong on pretty much every front - basically a collection of fact ls about anything matching Mendip, Nature, and Research.

However it did such a good job of pretending to know what it was talking about that I'm thinking of sending it a membership application form - it would fit in well at the club.

 

[alanw]

You only have to start doing stuff like cave and mine research to realise just how much material still isn't on the internet

Indeed. There's just been a request on our club FB page for the definition of "hading rift". Google can't find very many uses of the term, and no definitions whatsoever.

 

[Pitlamp]

You only have to start doing stuff like cave and mine research to realise just how much material still isn't on the internet

. . . which is precisely why caving libraries are so important.

;-)

 

[mikem]

However, hade / hading can be found (correct) on Wikipedia & several online dictionaries:

Hading

Definition, Synonyms, Translations of Hading by The Free Dictionary

www.thefreedictionary.com

 

[2xw]

I love it. It made significant improvements to my code. I asked it about the stuff that I am an expert in - I reckon it has a 2nd - 3rd year understanding of environmental science. However, there is lots of information online about the environment and also lots of information about code written in R. I also used it to write Excel macros because I can't be arsed to learn what those are written in, but again, plenty of excel stuff online.

It probably knows nothing about caving because, relatively, there is almost nothing about caving online. Be interesting to see what it comes up with if we trained an LLM on the contents of the BCL.


We will have to eventually admit that the vast, vast majority of human output isn't clever, imaginative, creative or interesting, it's just regurgitated, and it's with this stuff that it will be most useful. Ask it for a generic recipe or a fitness plan and it performs pretty well.

 

[mikem]

I think the problem may be that a hade is already a rift (fault or mineral vein), so a hading rift is a tautology

 

[langcliffe]

I think the problem may be that a hade is already a rift (fault or mineral vein), so a hading rift is a tautology

Hade is actually the angle of inclination of a fault / rift / lode, rather than the feature itself.

 

[Ian Ball]

Can it do calculus? Asking for a friend