Tagged: statistics Toggle Comment Threads | Keyboard Shortcuts

  • feedwordpress 08:01:02 on 2019/05/06 Permalink
    Tags: , , , Flash Crash, , , statistics, , ,   

    “There are three types of lies — lies, damn lies, and statistics”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    Charts

    “Hiding in Plain Sight”

     

    A chart’s purpose is usually to help you properly interpret data. But sometimes, it does just the opposite. In the right (or wrong) hands, bar graphs and pie charts can become powerful agents of deception, tricking you into inferring trends that don’t exist, mistaking less for more, and missing alarming facts. The best measure of a chart’s honesty is the amount of time it takes to interpret it, says Massachusetts Institute of Technology perceptual scientist Ruth Rosenholtz: “A bad chart requires more cognitive processes and more reasoning about what you’ve seen.”…

    Five examples (like the one above) of the kinds of tricks that charts can try to pull, explained: “Five Ways to Lie with Charts.”

    * Benjamin Disraeli

    ###

    As we stack the deck, we might recall that it was on this date in 2010, at 2:32p EDT, that the U.S. stock markets suffered a “Flash Crash”– in a period of just 36 minutes, the S&P 500, Dow Jones Industrial Average, and Nasdaq Composite collapsed and rebounded (the Dow, e.g., lost 9% of its value, then recovered most of it).

    Nearly five years later, the SEC charged a 36-year-old small-time trader who worked from his parents’ modest stucco house in suburban west London with having caused the collapse (using spoofing and layering, along with a form of front-running– all now explicitly outlawed).  But many experts are not convinced; to this day, there are numerous theories– but no consensus– as to the cause(s) of the crash.

    Flashcrash-2010

    The DJIA on May 6, 2010 (11:00 AM – 4:00 PM EDT)

    source

     

     
  • feedwordpress 08:01:42 on 2018/10/17 Permalink
    Tags: cause of death, , , Gertrude Mary Cox, , mortality, statistics,   

    “Life is pleasant. Death is peaceful. It’s the transition that’s troublesome.”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    shifting-death-preview-1

    Cause of death has changed over the years. In 1999, the suicide rate among 25- to 34-year-olds was 12.7 per 100,000 people. By 2016, that rate was almost 30 percent higher at 16.5.

    These shifts over time are common and vary across sex and age groups.

    With the release of the annual health report by the Centers for Disease Control and Prevention, I looked at the subcategories of mortality, as defined by the World Health Organization, focusing specifically on how the ten most common ways to die have changed over the years…

    causes of death

    See (a full-sized and working version of) Nathan Yau’s animation of the changing causes of death, by sex and age group, in the U.S. from 1999 to 2016: “Shifting Causes of Death.”

    * Isaac Asimov

    ###

    As we memento mori, we might spare a thoughts for Gertrude Mary Cox; she died on this date in 1978.  A pioneering statistician best known for her important work on experimental design, she founded the department of Experimental Statistics at North Carolina State University and later served as director of both the Institute of Statistics of the Consolidated University of North Carolina and the Statistics Research Division of North Carolina State University.  In 1949 Cox became the first female elected into the International Statistical Institute and in 1956 was President of the American Statistical Association.

    Siddell Studio source

     

     
  • feedwordpress 08:01:36 on 2017/07/20 Permalink
    Tags: , , Markov, Markov chains, Maryam Mirzakhani, , , statistics, stochastic processes,   

    “Mathematics, rightly viewed, possesses not only truth, but supreme beauty”*.. 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    Maryam Mirzakhani did not enjoy mathematics to begin with. She dreamed of being an author or politician, but as a top student at her all-girls school in Tehran she was still disappointed when her first-year maths exam went poorly. Her teacher believed her – wrongly – to have no particular affinity with the subject.

    Soon that would all change. “My first memory of mathematics is probably the time [my brother] told me about the problem of adding numbers from 1 to 100,” she recalled later. This was the story of Carl Gauss, the 18th-century genius whose schoolteacher set him this problem as a timewasting exercise – only for his precocious pupil to calculate the answer in a matter of seconds.

    The obvious solution is simple but slow: 1+2+3+4. Gauss’s solution is quicker to execute, and far more cunning. It goes like this: divide the numbers into two groups: from 1 to 50, and from 51 to 100. Then, add them together in pairs, starting with the lowest (1) and the highest (100), and working inwards (2+99, 3+98, and so on). There are 50 pairs; the sum of each pair is 101; the answer is 5050. “That was the first time I enjoyed a beautiful solution,” Mirzakhani told the Clay Mathematics Institute in 2008.

    Since then, her appreciation for beautiful solutions has taken her a long way from Farzanegan middle school. At 17 she won her first gold medal at the International Mathematics Olympiad. At 27 she earned a doctorate from Harvard University. The Blumenthal Award and Satter Prize followed, and in 2014 she became the first woman to be awarded the Fields Medal, the highest honour a mathematician can obtain.

    Before this particular brand of wonder became perceptible to Mirzakhani, she experienced feelings many of us can relate to: to the indifferent, her subject can seem “cold”, even “pointless”. Yet those who persist will be rewarded with glimpses of conceptual glory, as if gifted upon them by a capricious god: “The beauty of mathematics,” she warned, “only shows itself to more patient followers.”

    This concept of “beauty” found in maths has been referred to over centuries by many others; though, like beauty itself, it is notoriously difficult to define…

    For an experienced mathematician, the greatest equations are beautiful as well as useful. Can the rest of us see what they see?  “What makes maths beautiful?

    [From The New Humanist, via the ever-illuminating 3 Quarks Daily]

    Maryam Mirzakhani died last Friday, a victim of breast cancer; she was 40.  As Peter Sarnak (a mathematician at Princeton University and the Institute for Advanced Study) said, her passing is “a big loss and shock to the mathematical community worldwide.”  See also here.

    * Bertrand Russell, A History of Western Philosophy

    ###

    As we accede to awe, we might spare a thought for Andrey (Andrei) Andreyevich Markov; he died on this date in 1922.  A Russian mathematician, he helped to develop the theory of stochastic processes, especially those now called Markov chains: sequences of random variables in which the future variable is determined by the present variable but is independent of the way in which the present state arose from its predecessors.  (For example, the probability of winning at the game of Monopoly can be determined using Markov chains.)  His work on the study of the probability of mutually-dependent events has been developed and widely applied to the biological and social sciences.

     source

     

     
  • feedwordpress 08:01:18 on 2017/06/04 Permalink
    Tags: , , , milestone, , state of the world, statistics, , Worldometers, Zero Milestone   

    “The only lasting truth is Change”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    A running tally of world population, plus telling (and similarly constantly-updated) statistics on government and economics, society and media, the environment, food, water, energy, and health, all derived from sources including the United Nations Population Division, the World Health Organization (WHO), the Food and Agriculture Organization (FAO), the International Monetary Fund (IMF), and the World Bank: Worldometers.

    * Octavia E. Butler

    ###

    As we watch the world tick by, we might recall that it was on this date in 1923 that the Zero Milestone was dedicated just south of the White House at the north edge of the Ellipse, within President’s Park.  Intended as the initial milestone from which all road distances in the United States should be reckoned, at present only roads in the Washington, D.C. area have distances measured from it.

     source

     

     
  • feedwordpress 08:01:03 on 2017/04/10 Permalink
    Tags: , , , , , , random numbers, Sam Loyd, statistics,   

    “The generation of random numbers is too important to be left to chance”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    Roman 12mm dice

    Random numbers are central to more than we may realize.  They have applications in gambling, statistical sampling, computer simulation and Monte Carlo modeling, cryptography (as applied in both communications and transactions), completely randomized design, even sooth-saying– in any area where producing an unpredictable result is desirable.  So how they’re produced– the certainty that they are, in fact, random– matters enormously.

    It’s no surprise, then, that random number generation has a long and fascinating history.  Happily, Carl Tashian is here to explain.

    “As an instrument for selecting at random, I have found nothing superior to dice,” wrote statistician Francis Galton in an 1890 issue of Nature. “When they are shaken and tossed in a basket, they hurtle so variously against one another and against the ribs of the basket-work that they tumble wildly about, and their positions at the outset afford no perceptible clue to what they will be even after a single good shake and toss.”…

    From I Ching sticks and dice to the cryptographically-secure PRNG, “A Brief History of Random Numbers.”

    [TotH to the eminently-numerate Reuben Steiger]

    Robert Coveyou

    ###

    As we roll the bones, we might spare a thought for Samuel “Sam” Loyd; he died on this date in 1911. A chess player, chess composer, puzzle author, and recreational mathematician.  A member of the Chess Hall of Fame (for both his play and for his exercises, or “problems”), he gained posthumous fame when his son published a collection of his mathematical and logic puzzles, Cyclopedia of 5000 Puzzles after his father’s death.  As readers can see here and here, his puzzles still delight.

    Loyd’s most famous puzzle was the 14-15 Puzzle, which he produced in 1878. His original authorship is debated; but in any case, his version created a craze that swept America to such an extent that employers put up notices prohibiting playing the puzzle during office hours.

     source

     

     
  • feedwordpress 08:01:09 on 2017/03/15 Permalink
    Tags: , , , John Snow, , lying with statistics, , , statistics,   

    “Without data, you’re just another person with an opinion”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    It used to be that we’d see a poorly made graph or a data design goof, laugh it up a bit, and then carry on. At some point though — during this past year especially — it grew more difficult to distinguish a visualization snafu from bias and deliberate misinformation.

    Of course, lying with statistics has been a thing for a long time, but charts tend to spread far and wide these days. There’s a lot of them. Some don’t tell the truth. Maybe you glance at it and that’s it, but a simple message sticks and builds. Before you know it, Leonardo DiCaprio spins a top on a table and no one cares if it falls or continues to rotate.

    So it’s all the more important now to quickly decide if a graph is telling the truth…

    Nathan Yau (Flowing Data) provides a very helpful (and very amusing) guide: “How to Spot Visualization Lies.”

    * W. Edwards Deming

    ###

    As we key our eyes open, we might send healthy birthday greetings to John Snow; he was born on this date in 1813.  A physician and a leader in the adoption of anaesthesia and medical hygiene, he is considered the father of modern epidemiology, in large measure because of his work in tracing the source of a cholera outbreak in Soho, London, in 1854.  His On the Mode of Communication of Cholera (1849) suggested that cholera was a contagious disease easily transmitted by contaminated water. But the widely-held theory was that diseases are caused by bad air led to his idea being ignored.  Then, in London’s 1854 cholera emergency, he painstakingly correlated individual cholera casualties to the water supply they had used in each case.  He then communicated his results with a map that underlined his point, and ended the deadly epidemic by removing the pump handle of the community water pump that he found to be the culprit.

    Snow’s map of cholera cases

    source

    His findings inspired fundamental changes in the water and waste systems of London, which led to similar changes in other cities, and a significant improvement in general public health around the world.  His mode of communicating them contributed to the rise of data visualization.

     source

     

     
  • feedwordpress 09:01:12 on 2017/03/03 Permalink
    Tags: , , genetic drift, , , , , , Sewall Wright, statistics,   

    “When you have mastered numbers, you will in fact no longer be reading numbers, any more than you read words when reading books. You will be reading meanings.”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

    Errors of judgment about large numbers can have a big impact on the way you view policies and government decisions. The rationale goes like this: The National Science Foundation received $7.463 billion for fiscal year 2016 through the Consolidated Appropriations Act. The total United States budget outlay for 2016 was $3.54 trillion. If you’re someone who perceives the difference between a billion and a trillion as relatively small, you’d think the US is spending a lot of money on the National Science Foundation—in fact, depending on your politics, you might applaud the federal government’s investment or even think it wasteful. But, if you understand that a billion is a thousand times less than a trillion, you can calculate that the Foundation got a paltry 0.2 percent of the budget outlay last year. (It may be more straightforward to think of the budget as roughly one-half to one-third of reported costs for the proposed US-Mexico border wall, and let your values guide you from there.)…

    On the significance of scale: “How to Understand Extreme Numbers.

    [The image above is, of course, from the ever-wonderful xkcd.]

    * W.E.B. Du Bois

    ###

    As we nudge ourselves toward numeracy, we might spare a thought for Sewall Wright; he died on this date in 1988.  A geneticist, he was known for his influential work on evolutionary theory and also for his work on path analysis. He was a founder (with Ronald Fisher and J.B.S. Haldane) of population genetics– a major step in the development of the modern evolutionary synthesis combining genetics with evolution.   He is perhaps best remembered for his concept of genetic drift (called the Sewall Wright effect): when small populations of a species are isolated, the few individuals who carry certain relatively rare genes may fail, out of pure chance, to transmit them. The genes may therefore disappear and their loss may lead to the emergence of new species– although natural selection has played no part in the process.

     source

     

     
  • feedwordpress 09:01:34 on 2017/01/30 Permalink
    Tags: , , , , Roger N. Shepard, , statistics, , , William Davies   

    “A certain elementary training in statistical method is becoming as necessary for everyone living in this world of today as reading and writing”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    The declining authority of statistics – and the experts who analyse them – is at the heart of the crisis that has become known as “post-truth” politics. And in this uncertain new world, attitudes towards quantitative expertise have become increasingly divided. From one perspective, grounding politics in statistics is elitist, undemocratic and oblivious to people’s emotional investments in their community and nation. It is just one more way that privileged people in London, Washington DC or Brussels seek to impose their worldview on everybody else. From the opposite perspective, statistics are quite the opposite of elitist. They enable journalists, citizens and politicians to discuss society as a whole, not on the basis of anecdote, sentiment or prejudice, but in ways that can be validated. The alternative to quantitative expertise is less likely to be democracy than an unleashing of tabloid editors and demagogues to provide their own “truth” of what is going on across society.

    Is there a way out of this polarisation? Must we simply choose between a politics of facts and one of emotions, or is there another way of looking at this situation?One way is to view statistics through the lens of their history. We need to try and see them for what they are: neither unquestionable truths nor elite conspiracies, but rather as tools designed to simplify the job of government, for better or worse. Viewed historically, we can see what a crucial role statistics have played in our understanding of nation states and their progress. This raises the alarming question of how – if at all – we will continue to have common ideas of society and collective progress, should statistics fall by the wayside…

    The ability of statistics to represent the world accurately is declining. In its wake, a new age of big data controlled by private companies is taking over – and putting democracy in peril.  William Davies provides historical context, a clear diagnosis of the problem, and thoughts on a response in his important essay, “How statistics lost their power – and why we should fear what comes next.”

    * H.G. Wells, World Brain (1938)

    ###

    As we take note of numbers, we might send insightful birthday greetings to Roger Newland Shepard; he was born on this date in 1929.  A cognitive scientist and emeritus professor at Stanford, he has received both the National Medal of Science and the Rumelhart Prize.  While his contributions to his field are many, Shepard is probably best known as inventor of multidimensional scaling, a method for representing certain kinds of statistical data in a plane (or in space) with minimal distortion, so that the data can be apprehended by non-specialists.

     source

     

     
  • feedwordpress 09:01:04 on 2017/01/09 Permalink
    Tags: , Agnesi, , , , Jason Crease, , statistics, , witch of agnesi   

    “I go to seek a Great Perhaps”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    As we’ve noted before, 2016 seemed a bumper year for the Grim Reaper.  Jason Crease tested that perception against the data…

    It’s become cliché that unusually many prominent people died in 2016. Is this true?…

    Find out at “Was 2016 especially dangerous for celebrities? An empirical analysis.

    [Image above: source]

    * François Rabelais

    ###

    As we usher in the new, we might spare a thought for the first woman in the Western world considered to be a mathematician: Maria Gaetana Agnesi, she died this date in 1799.  While she thought and wrote broadly about natural science and philosophy, she is best remembered for her work in differential calculus– perhaps most particularly for her work on the cubic curve now know as the “witch of Agnesi.”

    source

     
  • feedwordpress 09:01:01 on 2017/01/02 Permalink
    Tags: , , , , living conditions, , McIlhenny, , statistics,   

    “Human progress is neither automatic nor inevitable”*… 


    Warning: preg_match_all(): Compilation failed: invalid range in character class at offset 7 in /homepages/23/d339537987/htdocs/pb/wp-content/themes/p2/inc/mentions.php on line 77

     

    … but happily, progress is made.

    As we fight hangovers (both from New Years festivities and from the slow-motion train wreck that was 2016), Max Roser reminds us that in many critical dimensions life has gotten better- much better– around the world… and he reminds us why it’s so very important that we understand this

    A history of global living conditions in 5 charts.”

    * “Human progress is neither automatic nor inevitable… Every step toward the goal of justice requires sacrifice, suffering, and struggle; the tireless exertions and passionate concern of dedicated individuals.”   ― Martin Luther King, Jr.

    ###

    As we look on the bright side, we might that it was on this date in 1890 that E.A. McIlhenny, the son of Tabasco brand pepper sauce inventor Edmund McIlhenny and manager of the family condiment empire, shot and killed a 19′ 1″ long alligator, reputedly the longest American alligator ever recorded.  McIlhenny, who was an amateur naturalist and conservationist, made the claim in one of his four books, The Alligator’s Life History.

     source

     

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
esc
cancel