Damn lies and statistics….

[MartinQ]

This was a government minister giving evidence at the Education Committee. I will happily concede that it’s unlikely to be a left/right problem (so not a political point!), more an example of a failure of education.

View attachment 608015

Sounds almost like a Yes Minister episode. Tbh, the Better than Average Effect is quite common
https://en.wikipedia.org/wiki/Illusory_superiority
Though in this case it sounds like political BS, which most politicians seem to suffer from.

 

[matticus]

Sounds he should have put another layer on his analysis and assigned priors to his assumptions, then used Bayes.
Then he could have got it wrong rigorously.

Back to the OP, though a slight detour from spurious correlations, one of my favourites is simpson's paradox
https://towardsdatascience.com/simpsons-paradox-and-interpreting-data-6a0443516765
Gets you thinking a bit ..

"lurking variables": I like that ...

 

[MartinQ]

"lurking variables": I like that ...

Its a bit like computer science which is a religion which believes any problem can be solved by another layer of abstraction.
If you've not able to solve a problem, you're just not abstracting/generalizing it correctly.

So we always need the mythical lurking variables (and Bayes) ....

 

[ClichéGuevara]

Its a bit like computer science which is a religion which believes any problem can be solved by another layer of abstraction.
If you've not able to solve a problem, you're just not abstracting/generalizing it correctly.

So we always need the mythical lurking variables (and Bayes) ....

I struggle with computing being a science in and of itself. It's simply a tool for the scientific process.

Because IT people work in data, quite a few seem to think that somehow makes them data analysts, or worse, statisticians. It's caused quite a few issues that I'm aware of.

 

[matticus]

I struggle with computing being a science in and of itself. It's simply a tool for the scientific process.

I suppose it's a bit like Maths in that sense - it's a pretty feckin important tool!
[and it needed a name, so 🤷‍♂️ ]

Because IT people work in data, quite a few seem to think that somehow makes them data analysts, or worse, statisticians. It's caused quite a few issues that I'm aware of.

Indeed (I think I said as much on Page1 )

 

[MartinQ]

I struggle with computing being a science in and of itself. It's simply a tool for the scientific process.

Because IT people work in data, quite a few seem to think that somehow makes them data analysts, or worse, statisticians. It's caused quite a few issues that I'm aware of.

I can't remember who said it / or the exact phrase (so it may just be me making it up ...), but someone in the 1950s said there was nothing to do with computer science that would be worth awarding a PhD for. Had a quick google, but too many PhD adverts for computer science came up :-)

You'd probably make a similar observation about maths (which isn't really a science), but as tools there both very useful.

Edit - beaten to it :-)

Edit 2 - Found it
https://www.google.co.uk/books/edit...that+merits+a+phd&pg=PR31&printsec=frontcover

 

[ClichéGuevara]

I suppose it's a bit like Maths in that sense - it's a pretty feckin important tool!
[and it needed a name, so 🤷‍♂️ ]


Indeed (I think I said as much on Page1 )

I totally agree, I just wish some of those in the field appreciated the difference between being IT literate, and understanding and interpreting data.

 

[MartinQ]

I totally agree, I just wish some of those in the field appreciated the difference between being IT literate, and understanding and interpreting data.

Agreed, this is pretty much the OP. Throw enough data at a algorithm and you're almost doomed to succeed if you're looking for strong relationships. Whether they're spurious is almost irrelevant .

 

[winjim]

Computer science is a mystery to me. I know how a programming language works and the very basics of writing code. I know how a transistor works and the very basics of electronic circuits. The bit in the middle, the interface between them, is pure witchcraft.

 

[mistyoptic]

Today’s Dilbert seems bang on topic

 

[twentysix by twentyfive]

Somewhere! It’s green….

Turned green from all that use