|
The new Global Cooling thread.
Bahamut.Milamber
Server: Bahamut
Game: FFXI
Posts: 3691
By Bahamut.Milamber 2014-06-24 15:03:12
You can correct different sections of data different ways, for different reasons; that is essential to do when you have different sources of different types. You generally are required to take the worst error margin of sampling, but that can be different based on how much the error for that dataset contributes to the overall sample.
Yes, I am extremely familiar with GIGO. That's generally why you even perform this in the first place, to even determine if you can draw any conclusions from the data, or to what extent. So let me ask you this, since it seems like we have come to an impasse.
How much data would have to be altered before the data set to be considered useless?
I think that a 5% error rate would signify issues in regards to the record keeping of the data, and anything above that would throw the entire set into question of feasibility. 6% error rate should automatically disqualify the data from usability.
Of course, the data we are talking about in question is greater than 6%.
What are your thoughts in that?
It depends on the cause for alteration. When using calibrated instruments, you could argue that anywhere from 0% to 100% of the actual value is altered, prior to the recording of data. In other words, the physical input to the system is altered in a potentially non-static but repeatable way for any given reading. That's not an error, but is an alteration of the input to the system to correct for known factors.
Frankly, what is an acceptable error rate depends on the application and sample rate versus the real input, and whether or not the sampling rate is significantly higher than the signal you are trying to reconstruct (see Nyquist rate).
If you have significant oversampling, you are quite tolerant to data errors or bad readings; hell, you could get have a 99% error rate and still reconstruct an original signal, as long as you have a method for reliably tossing data. The cost of that kind of stupidity is high; which is why you generally compromise with some sane error rate and corresponding sampling rate.
[+]
Server: Odin
Game: FFXI
Posts: 6558
By Odin.Zicdeh 2014-06-24 15:03:20
Bismarck.Ramyrez
Server: Bismarck
Game: FFXI
Posts: 4746
By Bismarck.Ramyrez 2014-06-24 15:03:26
I again reiterate my stance that regardless of whether you believe in the human impact upon climate change or not...where is the harm in cleaning up the environment, breathing cleaner air, and relying less and less on any fossil fuels, let alone foreign oil? No it won't be an immediate transition, but the people even trying to make the transition get hampered at every turn.
Bahamut.Milamber
Server: Bahamut
Game: FFXI
Posts: 3691
By Bahamut.Milamber 2014-06-24 15:05:15
You seem to have unrealistic expectations of data accuracy spanning the types and timeframe that climate data does. Wait, it's more advanced than "The temperature at XXX was XX degrees as the high, recorded at X:XX, XX degrees as the low, recorded at X:XX, the barometric pressure was XXX, humidity was at XXX, it was <insert weather> for XX hours, <insert weather> for XX hours, (so on), and XX amount of rain was collected"?
You know, something that is recorded at every airport in the country on a daily basis (at the very least).
Oh, I forgot, there's "climate speech" included in that data that only "climate scientists" would know about.
Also very little understanding of how data is analyzed. No, that's you bub. Dear effing god yes, it is more complicated than that.
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:16:17
It depends on the cause for alteration. When using calibrated instruments, you could argue that anywhere from 0% to 100% of the actual value is altered, prior to the recording of data. In other words, the physical input to the system is altered in a potentially non-static but repeatable way for any given reading. That's not an error, but is an alteration of the input to the system to correct for known factors. That shouldn't effect the data collection as the calibration is used to make sure that the system in place is using the same guidelines of recording the data.
If people argue against calibrated systems, I could argue against uncalibrated systems, based on wear and tear of the tools and systems in place after use.
Frankly, what is an acceptable error rate depends on the application and sample rate versus the real input, and whether or not the sampling rate is significantly higher than the signal you are trying to reconstruct (see Nyquist rate).
If you have significant oversampling, you are quite tolerant to data errors or bad readings; hell, you could get have a 99% error rate and still reconstruct an original signal, as long as you have a method for reliably tossing data. The cost of that kind of stupidity is high; which os why you generally compromise with some sane error rate and corresponding sampling rate. Statistically speaking.
But if the data requires a point in time recording (it was XX degrees on MM/DD/YYYY) then there should be little error in the process. We are talking about simple data that is recorded by a simple system, and yet, with these two agencies, a huge discrepancy between raw and adjusted data.
You have only 365/366 data entry points per year. It should not be hard to record such data, a 3rd grader could do it constantly and without errors.
Please don't forget what type of data we are talking about here.
I again reiterate my stance that regardless of whether you believe in the human impact upon climate change or not...where is the harm in cleaning up the environment, breathing cleaner air, and relying less and less on any fossil fuels, let alone foreign oil? No it won't be an immediate transition, but the people even trying to make the transition get hampered at every turn. Nothing wrong with that when we have the technology to feasibly use alternative fuel sources and for it to be cheaper and equal power to current fossil fuels.
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:17:39
You seem to have unrealistic expectations of data accuracy spanning the types and timeframe that climate data does. Wait, it's more advanced than "The temperature at XXX was XX degrees as the high, recorded at X:XX, XX degrees as the low, recorded at X:XX, the barometric pressure was XXX, humidity was at XXX, it was <insert weather> for XX hours, <insert weather> for XX hours, (so on), and XX amount of rain was collected"?
You know, something that is recorded at every airport in the country on a daily basis (at the very least).
Oh, I forgot, there's "climate speech" included in that data that only "climate scientists" would know about.
Also very little understanding of how data is analyzed. No, that's you bub. Dear effing god yes, it is more complicated than that. How come the basis of every "climate change" argument is average temperature per year is rising?
There would need to be more data than that to discover why, but the argument is what.
Cerberus.Pleebo
Server: Cerberus
Game: FFXI
Posts: 9720
By Cerberus.Pleebo 2014-06-24 15:22:42
Thanks for confirming that you did not bother to read the document I linked at all. I mean, it was probably a given in the first place, but it was getting onerous giving you the benefit of the doubt.
Lakshmi.Zerowone
Server: Lakshmi
Game: FFXI
Posts: 6949
By Lakshmi.Zerowone 2014-06-24 15:24:56
Breaking news CPA moonlights as Climatologist on the Internet.
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:27:31
Thanks for confirming that you did not bother to read the document I linked at all. What document?
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:27:48
Breaking news CPA moonlights as Climatologist on the Internet. Who said I was trying to be a Climatologist?
I questioned the "scrubbed" data, that is the whole argument. I presented my arguments and only a couple of people presented counterarguments. Couple others were being idiots, but that's no surprise coming from them.
I did not write a thesis or conduct a study in climatology. So, where do you even remotely get the idea that I attempted to do so?
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:28:38
Thanks for confirming that you did not bother to read the document I linked at all. What document? This one.
The one that Pleebo can't understand.
[+]
Cerberus.Pleebo
Server: Cerberus
Game: FFXI
Posts: 9720
By Cerberus.Pleebo 2014-06-24 15:30:15
I understand most of it just fine. I'm awaiting your detailed criticisms on their bias corrections.
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:33:46
Thanks for confirming that you did not bother to read the document I linked at all. What document? This one.
The one that Pleebo can't understand. Haha, that's what I thought.
Cerberus.Pleebo
Server: Cerberus
Game: FFXI
Posts: 9720
By Cerberus.Pleebo 2014-06-24 15:34:59
Oh good, Chaos. King seems to be having trouble articulating his specific criticisms of the USHCN bias corrections. Perhaps you can share some of your insights?
Bahamut.Milamber
Server: Bahamut
Game: FFXI
Posts: 3691
By Bahamut.Milamber 2014-06-24 15:35:44
Lets just say you use a simple pt100 sensor for temperature. What happens if someone places a structure nearby, and the windows during certain parts of the day where you happen to sample reflect light onto the sensor/sensor housing. Repeat ad nauseum for all kinds of reflective sources.
Perhaps you have an issue with regards to shielding or the ground plane on the PCB, and are having SPI communication errors giving you erroneous readings because the microcontroller is reading a double clock cycle when someone uses an arc welder.
Perhaps the sensor has had too much condensation or rain/freeze cycles, and now has a thick coating of ice.
Perhaps birds have found a nice place to make a nest.
Perhaps birds have found a good place to perch overhead, and give the sensor a regular coating of excrement, leading towards ever slower transitions towards ambient temperature.
Perhaps you live in a place where people like to vandalize or shoot things, and have damaged the station.
Perhaps that third grader has been taking temperatures at different times, or different places, or with different thermometers.
Doing things consistently, correctly, and accurately in all circumstances is HARD.
[+]
Bahamut.Ravael
Server: Bahamut
Game: FFXI
Posts: 13640
By Bahamut.Ravael 2014-06-24 15:36:15
Gotta say, I'm having a good laugh at you guys trying to argue about statistics.
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:39:07
I wouldn't do that Chaos.
Pleebo is looking for any argumentative answer against the methodology of his report.
There is nothing wrong with the methodology, which I have stated two separate times that there is nothing wrong with it.
He just trying to goad you into a pointless fight he thinks he can win.
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:39:34
I wouldn't do that Chaos.
Pleebo is looking for any argumentative answer against the methodology of his report.
There is nothing wrong with the methodology, which I have stated two separate times that there is nothing wrong with it.
He just trying to goad you into a pointless fight he thinks he can win.
Basically he's saying don't worry about it, it's all a scam anyway so he can get some of the sweet climate change research grant money. But you gotta play the part in order to get accepted for that money.
[+]
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:41:13
Also one must posses a degree from a college or university too.
[+]
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:42:45
Like I said in other threads, I believe the experts with real degrees from the IPCC. The people who took the time to gather the data themselves.
Cerberus.Pleebo
Server: Cerberus
Game: FFXI
Posts: 9720
By Cerberus.Pleebo 2014-06-24 15:43:15
No, you are arguing that bias correction beyond some arbitrary threshold renders a data set garbage. So tell me, Chaos or King, what about the bias corrections detailed in the report I linked leads you to believe that the corrected USHCN data is not usable?
Like I said in other threads, I believe the experts with real degrees from the IPCC. So then you agree with me. Guess I only have to direct my questions towards King now.
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:44:25
Lets just say you use a simple pt100 sensor for temperature. What happens if someone places a structure nearby, and the windows during certain parts of the day where you happen to sample reflect light onto the sensor/sensor housing. Repeat ad nauseum for all kinds of reflective sources. Pleebo's article states some discrepancies arise due to movement of test locations.
That's to be accounted for. That isn't an error, but a prevention of an error to get better data. The previous data isn't compromised.
Perhaps the sensor has had too much condensation or rain/freeze cycles, and now has a thick coating of ice.
Perhaps birds have found a nice place to make a nest.
Perhaps birds have found a good place to perch overhead, and give the sensor a regular coating of excrement, leading towards ever slower transitions towards ambient temperature.
Perhaps you live in a place where people like to vandalize or shoot things, and have damaged the station. Unless there is lack of maintenance and/or equipment review, I don't think that would be a big issue.
If nobody checks on the equipment and/or performs regular maintenance/calibration, then the data will obviously be skewed/tainted. That would be recording error, not equipment error.
Doing things consistently, correctly, and accurately in all circumstances is HARD. I understand that. But if people are going to swear by the data, then the data should be verified.
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:45:53
Like I said in other threads, I believe the experts with real degrees from the IPCC. The people who took the time to gather the date themselves. It's not the experts that I'm questioning though.
It's the data.
[+]
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:48:08
So then you agree with me. Guess I only have to direct my questions towards King now. Not exactly, you argued with everything they said in their video discussions. Unless you know better now?
You just want people to agree with you no matter what you say. It gets old fast.
Cerberus.Pleebo
Server: Cerberus
Game: FFXI
Posts: 9720
By Cerberus.Pleebo 2014-06-24 15:48:56
IPCC folks were likely not involved in the data collection lol. They're not a research organization. How can you feel qualified to comment on these things when you don't even know the basics?
Not exactly, you argued with everything they said in their video discussions. Unless you know better now?
You just want people to agree with you no matter what you say. It gets old fast. So you agree with the 2 people who used to be associated with the IPCC that disagree with current model projections rather than the vast majority who support them. The *** is this?
You know what gets old? People who constantly reinforce their ignorance every time they open their mouths about something they have zero understanding of constantly commenting on or creating these topics and spreading misinformation.
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:50:00
Like I said in other threads, I believe the experts with real degrees from the IPCC. The people who took the time to gather the date themselves. It's not the experts that I'm questioning though.
It's the data. It's more the data and how it relates to all fields of science. That's why they have scientists from all different backgrounds who try to paint a bigger picture.
Leviathan.Chaosx
Server: Leviathan
Game: FFXI
Posts: 20284
By Leviathan.Chaosx 2014-06-24 15:50:52
IPCC folks were likely not involved in the data collection lol. They're not a research organization. How can you feel qualified to comment on these things when you don't even know the basics? Right coming from the guy who doesn't even have an education.
[+]
Bahamut.Ravael
Server: Bahamut
Game: FFXI
Posts: 13640
By Bahamut.Ravael 2014-06-24 15:52:21
I'm going to go out on a limb and say that the people taking the measurements in general didn't use control charts to monitor their processes. Data cleaning becomes a huge mess when measuring equipment gets out of statistical control. Frequent calibration is very important.
[+]
Server: Asura
Game: FFXI
Posts: 34187
By Asura.Kingnobody 2014-06-24 15:52:24
Like I said in other threads, I believe the experts with real degrees from the IPCC. The people who took the time to gather the date themselves. It's not the experts that I'm questioning though.
It's the data. It's more the data and how it relates to all fields of science. That's why they have scientists from all different backgrounds who try to paint a bigger picture. They can only work with what they are given, though.
These are supposed to be historical data, but Alt showed reasonable evidence to show that the historical data could be corrupted due to too much alterations and adjustments.
[+]
By fonewear 2014-06-24 15:52:40
IPCC folks were likely not involved in the data collection lol. They're not a research organization. How can you feel qualified to comment on these things when you don't even know the basics? Right coming from the guy who doesn't even have an education.
I suggest we "create education better"
YouTube Video Placeholder
[+]
Turns out the US has been cooling sense the 1930's when you look at charts/graphs that have not been "altered". Looks like I'm not the only one that has a problem with fudged data.
http://stevengoddard.wordpress.com/data-tampering-at-ushcngiss/
|
|