Showing posts with label scientific method. Show all posts
Showing posts with label scientific method. Show all posts

Thursday, 20 March 2008

Limitations of the 'Scientific Method'

--- In bhascience@yahoogroups.com, "david_c_flint" wrote:
>
> I'm glad you asked me that! I will upload a paper I wrote two years
> ago giving one approach.
>
> dcf
>
>
> --- In bhascience@yahoogroups.com, "Chris Street" chris@ wrote:
> ...
>
> > So what ARE the limitations of the Scientific Method?


David, I think one of the most important limitations of the Scientific Method is (as you say in your upload file) "research often delivers qualified answers whereas citizens and policy makers want definitive answers"

I think terms like 'probably' or 'highly likely', when discussing science, can be used with bullish confidence.

As a corollary, the strength of the scientific method is that current research is provisional and if falsified may be replaced by science that better describes how the world works.

I was very impressed with the 2007 Intergovernmental Panel on Climate Change (IPCC ) final report (pdf) which I reviewed in my Climate Alternative Temperature Science (CATS) blog.

The IPCC considered about 29,000 pieces of real-world evidence and concluded:


Uncertainty
  • climate change is "unequivocal"
  • Probable temperature rise between 1.8C and 4C
  • Possible temperature rise between 1.1C and 6.4C
Likelyhood
  • humankind's emissions of greenhouse gases are more than 90% very likely to be the main cause
  • Sea level most likely to rise by 28-43cm
  • Increase in heat waves very likely
  • Increase in tropical storm intensity likely
Confidence
  • Changes in snow, ice and frozen ground have with high confidence increased the number and size of glacial lakes
  • earlier timing of spring events and poleward and upward shifts in plant and animal ranges
    are with very high confidence linked to recent warming.
IPCC defined what they meant by the above terms in red in the IPPC glossary:-

Uncertainty
An expression of the degree to which a value (e.g., the future state of the climate system) is unknown. Uncertainty can result from lack of information or from disagreement about what is known or even knowable. Uncertainty can therefore be represented by quantitative measures, for example, a range of values calculated by various models, or by qualitative statements, for example, reflecting the judgement of a team of experts.

Likelihood
The likelihood of an occurrence, an outcome or a result, where this can be estimated probabilistically, is expressed in IPCC reports using a standard terminology defined as follows:

Terminology Likelihood of the occurrence / outcome
Virtually certain >99% probability of occurrence
Very likely >90% probability
Likely >66% probability
More likely than not >50% probability
About as likely as not 33 to 66% probability
Unlikely <33% probability
Very unlikely <10% probability
Exceptionally unlikely <1% probability

Confidence
The level of confidence in the correctness of a result is expressed in this report, using a standard terminology defined as follows:

Terminology Degree of confidence in being correct
Very high confidence = At least 9 out of 10 chance of being correct
High confidence = About 8 out of 10 chance
Medium confidence = About 5 out of 10 chance
Low confidence = About 2 out of 10 chance
Very low confidence = Less than 1 out of 10 chance

I think this measured language in terms of uncertainty, likelyhood and confidence brilliantly conveys the provisional nature of the scientific method to non-scientists and politicians.

Friday, 11 May 2007

Working Group I - Technical Summary - Treatment of Uncertainties

Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change - Technical Summary

Box TS.1: T
reatment of Uncertainties in the Working Group I Assessment
The importance of consistent and transparent treatment of uncertainties is clearly recognised by the IPCC in preparing its assessments of climate change. The increasing attention given to formal treatments of uncertainty in previous assessments is addressed in Section 1.6. To promote consistency in the general treatment of uncertainty across all three Working Groups,
authors of the Fourth Assessment Report have been asked to follow a brief set of guidance notes on determining and describing uncertainties in the context of an assessment.

2 This box summarises the way that Working Group I has applied those guidelines and covers some aspects of the treatment of uncertainty specific to material assessed here. Uncertainties can be classified in several different ways according to their origin. Two primary types are ‘value uncertainties’ and ‘structural uncertainties’. Value uncertainties arise from the incomplete determination of particular values or results, for example, when data are inaccurate or not fully representative of the phenomenon of interest. Structural uncertainties
arise from an incomplete understanding of the processes that control particular values or results, for example, when the conceptual framework or model used for analysis does not include all the relevant processes or relationships.

Value uncertainties are generally estimated using statistical techniques and expressed probabilistically. Structural uncertainties are generally described by giving the authors’ collective judgment of their confidence in the correctness of a result. In both cases, estimating uncertainties is intrinsically about describing the limits to knowledge and for this reason involves expert judgment about the state of that knowledge. A different type of uncertainty arises in systems that are either chaotic or not fully deterministic in nature and this also limits our ability to project all aspects of climate change.

The scientific literature assessed here uses a variety of other generic ways of categorising uncertainties. Uncertainties associated with ‘random errors’ have the characteristic of decreasing as additional measurements are accumulated, whereas those associated with ‘systematic errors’ do not. In dealing with climate records, considerable attention has been
given to the identification of systematic errors or unintended biases arising from data sampling issues and methods of analysing and combining data. Specialised statistical methods based on quantitative analysis have been developed for the detection and attribution of climate change and for producing probabilistic projections of future climate parameters. These are summarised in the relevant chapters.

The uncertainty guidance provided for the Fourth Assessment Report draws, for the first time, a careful distinction between levels of confidence in scientific understanding and the likelihoods of specific results. This allows authors to express high confidence that an event is extremely unlikely (e.g., rolling a dice twice and getting a six both times), as well as high confidence that an event is about as likely as not (e.g., a tossed coin coming up heads). Confidence and likelihood
as used here are distinct concepts but are often linked in practice.

The standard terms used to define levels of confidence in this report are as given in the IPCC Uncertainty Guidance Note, namely:

Confidence Terminology Degree of confidence in being correct
Very high confidence At least 9 out of 10 chance
High confidence About 8 out of 10 chance
Medium confidence About 5 out of 10 chance
Low confidence About 2 out of 10 chance
Very low confidence Less than 1 out of 10 chance

Note that ‘low confidence’ and ‘very low confidence’ are only used for areas of major concern and where a risk-based perspective is justified.

Chapter 2 of this report uses a related term ‘level of scientific understanding’ when describing uncertainties in different contributions to radiative forcing. This terminology is used for consistency with the Third Assessment Report, and the basis on which the authors have determined particular levels of scientific understanding uses a combination of approaches
consistent with the uncertainty guidance note as explained in detail in Section 2.9.2 and Table

2.11.
The standard terms used in this report to define the likelihood of an outcome or result where this can be estimated probabilistically are:

Likelihood Terminology Likelihood of the occurrence/ outcome
Virtually certain > 99% probability
Extremely likely > 95% probability
Very likely > 90% probability
Likely > 66% probability
More likely than not > 50% probability
About as likely as not 33 to 66% probability
Unlikely < 33% probability
Very unlikely < 10% probability
Extremely unlikely < 5% probability
Exceptionally unlikely < 1% probability

The terms ‘extremely likely’, ‘extremely unlikely’ and ‘more likely than not’ as defined above have been added to those given in the IPCC Uncertainty Guidance Note in order to provide a more specific assessment of aspects including attribution and radiative forcing.

Unless noted otherwise, values given in this report are assessed best estimates and their uncertainty ranges are 90% confidence intervals (i.e., there is an estimated 5% likelihood of the value being below the lower end of the range or above the upper end of the range). Note that in some cases the nature of the constraints on a value, or other information available, may indicate an asymmetric distribution of the uncertainty range around a best estimate. In such cases, the uncertainty range is given in square brackets following the best estimate.

Sunday, 6 May 2007

Intergovernmental Panel on Climate Change - Climate Change 2007

Climate Change 2007: The Physical Science Basis - Contribution of Working Group I (WG1) to the Fourth Assessment. Published February 2007.

News Release: IPCC adopts major assessment of climate change science

Paris, 2 February 2007 – Late last night, Working Group I of the Intergovernmental Panel on Climate Change (IPCC) adopted the Summary for Policymakers of the first volume of “Climate Change 2007”, also known as the Fourth Assessment Report (AR4).

“Climate Change 2007: The Physical Science Basis”, assesses the current scientific knowledge of the natural and human drivers of climate change, observed changes in climate, the ability of science to attribute changes to different causes, and projections for future climate change.

The report was produced by some 600 authors from 40 countries.

In this Summary for Policymakers the following levels of confidence have been used to express expert judgments on the correctness of the underlying science:
very high confidence - at least a 9 out of 10 chance of being correct;
high confidence - about an 8 out of 10 chance of being correct.

The following terms have been used to indicate the assessed likelihood, using expert judgement, of an outcome or a result:
Virtually certain > 99% probability of occurrence,
Extremely likely > 95%,
Very likely > 90%,
Likely > 66%,
More likely than not > 50%,
Unlikely less than 33%,
Very unlikely less than 10%,
Extremely unlikely less than 5%.

In general, uncertainty ranges for results given in this Summary for Policymakers are 90% uncertainty intervals unless stated otherwise, i.e., there is an estimated 5% likelihood that the value could be above the range given in square brackets and 5% likelihood that the value could be below that range.

* the Intergovernmental Panel on Climate Change (IPCC) assesses scientific, technical and socio- economic information relevant for the understanding of climate change, its potential impacts and options for adaptation and mitigation.

* Global atmospheric concentrations of carbon dioxide, methane and nitrous oxide have increased markedly as a result of human activities since 1750 and now far exceed pre-industrial values (Fig. SPM-1).

* Global increases in carbon dioxide concentration are due primarily to fossil fuel use and land-use change, while those of methane and nitrous oxide are primarily due to agriculture.

* Carbon dioxide is the most important anthropogenic (def: Caused by humans) greenhouse gas. The global atmospheric concentration of carbon dioxide has increased from a pre-industrial value of about 280 ppm to 379 ppm in 2005.

* The atmospheric concentration of carbon dioxide in 2005 exceeds by far the natural range over the last 650,000 years (180 to 300 ppm) as determined from ice cores.

* The annual carbon dioxide concentration growth-rate was larger during the last 10 years (1995 – 2005 average: 1.9 ppm per year), than it has been since the beginning of continuous direct atmospheric measurements (1960 – 2005 average: 1.4 ppm per year).

* Warming of the climate system is unequivocal, as is now evident from observations of increases in global average air and ocean temperatures, widespread melting of snow and ice, and rising global average sea level.

* Observations since 1961 show that the average temperature of the global ocean has increased to depths of at least 3000 m and that the ocean has been absorbing more than 80% of the heat added to the climate system. Such warming causes seawater to expand, contributing to sea level rise.

* The average atmospheric water vapour content has increased since at least the 1980s over land and ocean as well as in the upper troposphere. The increase is broadly consistent with the extra water vapour that warmer air can hold.

* Mountain glaciers and snow cover have declined on average in both hemispheres.

* Widespread decreases in glaciers and ice caps have contributed to sea level rise (ice caps do not include contributions from the Greenland and Antarctic ice sheets). Losses from the ice sheets of Greenland and Antarctica have very likely contributed to sea level rise over 1993 to 2003.

* Global average sea level rose at an average rate of 1.8 mm per year over 1961 to 2003. The rate was faster over 1993 to 2003, about 3.1 mm per year.

* There is high confidence that the rate of observed sea level rise increased from the 19th to the 20th century. The total 20th century rise is estimated to be 0.17 m.

* Arctic temperatures increased at almost twice the global average rate in the past 100 years.

* since 1978 annual average Arctic sea ice extent has shrunk by 2.7% per decade, with larger decreases in summer of 7.4% per decade.

* the warmth of the last half century is unusual in at least the previous 1300 years. The last time the polar regions were significantly warmer than present for an extended period (about 125,000 years ago), reductions in polar ice volume led to 4-6 metres of sea level rise.

* Northern Hemisphere temperatures during the second half of the 20th century were very likely higher than during any other 50-year period in the last 500 years and likely the highest in at least the past 1300 years.

* Global average sea level in the last interglacial period (about 125,000 years ago) was likely 4 to 6 m higher than during the 20th century, mainly due to the retreat of polar ice.

* Ice core data indicate that polar temperatures at that time were 3 to 5°C higher than present, because of differences in the Earth’s orbit.

* The Greenland ice sheet and other Arctic ice fields likely contributed no more than 4 m of the observed sea level rise.

* Most of the observed increase in globally averaged temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations.

* Discernible human influences now extend to other aspects of climate, including ocean warming, continental-average temperatures, temperature extremes and wind patterns (see Figure SPM-4 and Table SPM-2).

* global average surface warming following a doubling of carbon dioxide concentrations is likely to be in the range 2 to 4.5°C with a best estimate of about 3°C, and is very unlikely to be less than 1.5°C. Values substantially higher than 4.5°C cannot be excluded.

* Water vapour changes represent the largest feedback affecting climate sensitivity and are now better understood than in the TAR.

* Cloud feedbacks remain the largest source of uncertainty.

* It is very unlikely that climate changes of at least the seven centuries prior to 1950 were due to variability generated within the climate system alone. A significant fraction of the reconstructed Northern Hemisphere interdecadal temperature variability over those centuries is very likely attributable to volcanic eruptions and changes in solar irradiance, and it is likely that anthropogenic forcing contributed to the early 20th century warming evident in these records.

* Continued greenhouse gas emissions at or above current rates would cause further warming and induce many changes in the global climate system during the 21st century that would very likely be larger than those observed during the 20th century.

* Sea ice is projected to shrink in both the Arctic and Antarctic. In some projections, Arctic late-summer sea ice disappears almost entirely by the latter part of the 21st century.

* It is very likely that hot extremes, heat waves, and heavy precipitation events will continue to become more frequent.

* it is likely that future tropical cyclones (typhoons and hurricanes) will become
more intense, with larger peak wind speeds and more heavy precipitation associated with ongoing increases of tropical SSTs.

* Increases in the amount of precipitation are very likely in high-latitudes, while decreases are likely in most subtropical land regions

* it is very likely that the meridional overturning circulation (MOC) of the
Atlantic Ocean will slow down during the 21st century. Temperatures in the Atlantic region are projected to increase due to the much larger warming associated with projected increases of greenhouse gases. It is very unlikely that the MOC will undergo a large abrupt transition during the 21st century.

* Anthropogenic warming and sea level rise would continue for centuries due to the timescales associated with climate processes and feedbacks, even if greenhouse gas concentrations were to be stabilized.

* Climate carbon cycle coupling is expected to add carbon dioxide to the atmosphere as the climate system warms, but the magnitude of this feedback is uncertain.

* Contraction of the Greenland ice sheet is projected to continue to contribute to sea level rise after 2100.

* Current global model studies project that the Antarctic ice sheet will remain too cold for widespread surface melting and is expected to gain in mass due to increased snowfall. However, net loss of ice mass could occur if dynamical ice discharge dominates the ice sheet mass balance.

* Both past and future anthropogenic carbon dioxide emissions will continue to contribute to warming and sea level rise for more than a millennium, due to the timescales required for removal of this gas from the atmosphere.

Behind the scenes at the Paris climate change meeting

click for 9th February Podcast (9min to 17 minutes)
  • IPCC final 2007 report (2nd Feb released) Scientist & Governmental concensus report says 90% (9/10) chance (ie. very likely) that climate change is caused by man
    • Chinese & Saudia Arabia delegation argued that their was a 66% chance (likely) that climate change is caused by man. NB. The "likely" term was used in the final IPCC 2001 report. The Chinese arguement was dismissed in the final 2007 IPCC report.
    • US delegation was cooperative and constructive
  • unequivocal evidence that global warming is happening

reposted from: NS podcast
my: highlights / emphasis / key points / comments

Scientific Method used in IPCC Climate Change report

The Intergovernmental Panel on Climate Change (IPCC) report was issued February 2007. Certain terms are used and defined to represent scientifically objective statements (eg Very Likely >90% chance).

In general, uncertainty ranges for results given in this Summary for Policymakers are 90% uncertainty intervals unless stated otherwise, i.e., there is an estimated 5% likelihood that the value could be above the range given in square brackets and 5% likelihood that the value could be below that range. Best estimates are given where available. Assessed uncertainty intervals are not always symmetric about the corresponding best estimate. Note that a number of uncertainty ranges in the Working Group I TAR corresponded to 2-sigma (95%), often using expert judgement. (page 2)

In this Summary for Policymakers, the following terms have been used to indicate the assessed likelihood, using expert judgement, of an outcome or a result (page 4):

  • Virtually certain > 99% probability of occurrence
  • Extremely likely > 95%
  • Very likely > 90%
  • Likely > 66%,
  • More likely than not > 50%
  • Unlikely <33%>
  • Very unlikely <10%
  • Extremely unlikely <5%
Examples
  • Most of the observed increase in globally averaged temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations. This is an advance since the TAR’s 2001 conclusion that “most of the observed warming over the last 50 years is likely to have been due to the increase in greenhouse gas concentrations”.

In this Summary for Policymakers the following levels of confidence (page 5) have been used to express expert judgments on the correctness of the underlying science:
  • very high confidence at least a 9 out of 10 chance of being correct
  • high confidence about an 8 out of 10 chance of being correct.
Examples
  • The understanding of anthropogenic warming and cooling influences on climate has improved since the Third Assessment Report (TAR), leading to very high confidence that the globally averaged net effect of human activities since 1750 has been one of warming
Unequivocal example: Warming of the climate system is unequivocal, (not defined) as is now evident from observations of increases in global average air and ocean temperatures, widespread melting of snow and ice, and rising global average sea level.

Reference: Intergovernmental Panel on Climate Change IPCC pdf report