reckless intuitions of an epistemic hygienist's LiveJournal Entries [entries|friends|calendar]
reckless intuitions of an epistemic hygienist

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ calendar | livejournal calendar ]

[09 Feb 2020|05:47pm]
Wouldn't you like to read my academic posts instead? Stats Computer Science

LiveJournal revival!! [19 Dec 2016|10:57pm]
LiveJournal has been a powerful force in my life, as it has been for many people of my demographic (English-speaking nerds born in 1975-1985). It started winding down in 2010. In the last couple of years, I've been talking about the idea of Christmas-time revival, and to my happy surprise someone else has decided to make it happen. Maybe it will stick this time, though few of my friends can spare the time nowadays and many of us now get our social needs met IRL (especially those of us who came to the Bay Area).

mirror of this post
11 comments   | comment & sign your name

lit review: Kullback-Leibler divergence in Statistics PhD textbooks [13 Dec 2015|04:27pm]
Bickel, Klaassen, Ritov, Wellner
* p. 227, Re: estimating cell probabilities in a two-way contingency table with known marginals.  "Iterative Proportional Fitting converges to the minimum Kullback divergence estimator"

Casella & Berger (2nd Edition) says nothing

Gelman, Carlin, Stern, Rubin (2nd Edition)
* p.107, asymptotics of estimation under mis-specified model.
* p.586-588, details about this.

Hastie, Tibshirani, Friedman
 (2nd Edition)
* p. 561, KL "distance".  Used in definition of mutual information, on a section about ICA.

Hogg, McKean, Craig (6th edition) says nothing

* p.113-114, "Kullback-Leibler information number", on chapter titled "Strong Consistency of Maximum-Likelihood Estimates"

* p.59: proof question: show that KL > 0 unless the two distributions are equal.
* p.156: consistency of MLE.
* p.466: solution to the proof question.

Lehmann & Romano
* p. 432: optimal test rejects for large values of T_n, and T_n converges to -K(P0, P1).
* p. 672: in non-parametric test for the mean, KL is used to define the most favorable distribution in H0.

van der Vaart
* p. 56: asymptotics of estimation under mis-specified model.
* p. 62: viewing MLE as an M-estimator.

* p. 125: consistency of MLE. "By LLN, M_n converges to -D(θ*, θ)"
mirror of this post
comment & sign your name

R <-> Julia dictionary [12 Mar 2014|02:32pm]

I welcome improvements and additions to this.

* words in ALLCAPS denote macros

* x denotes arbitrary variables

* a, b denote scalars

* f, g denote functions

* i,j denote indices

* mat denotes matrices

* obj denotes objects 

* v, w denote vectors

* z denotes a Boolean truth value


mirror of this post
1 comment   | comment & sign your name

yoga and intersubjectivity: let's invent a language [12 Feb 2014|01:40am]
(originally posted Oct 25, 2012, 01:40)

Perhaps one of the defining traits of "nerds" is a low level of body awareness, which comes with "spending too much time in the head". This may explain why yoga has been so revealing for me. I have been learning which sensations correspond to stretch, strain, and pain; and how to move muscles independently of other muscles (often my brain used to think of them as just one thing). Sometimes I need visual feedback to learn to control my muscles. I am lucky to have a teacher who understands how unintuitive this is for me.

I wish we had a standard language for naming specific sensations. I would like to convey precisely the twinge on my lower back, which might be a pinched nerve, but might just be soreness. If my teacher could feel what I feel, he would know what it was, but instead his judgement has to rely on my imperfect attempts at describing it.

When it comes to bodily sensations, we don't know how much subjectivity there is. Psychologists (psychophysicists) can often quantify the subjectivity of senses (say color), because even when words fail, they can do experiments to test whether subjects are able to detect tiny differences in stimuli (perhaps defining a metric on perceptual space, or more!), and then quantify how much people differ in this ability, in different regions of stimulus space. But when it comes to your body, it is much harder to stimulate a sensation to a precision worthy of being called "reproducible". And then there's habituation (which is also a problem for scientists trying to study smell).

Right now you could start a philosophical food fight by bringing up the label-switching problem (namely that, just because you and your teacher are in verbal agreement doesn't mean that your experiences agree), but I just want to be practical here: how can we develop a shared vocabulary that would allow me to better convey my sensation to my teacher, so that he may make a better guess about what is wrong with my back? Are there existing human cultures in which people can easily convey their bodily sensations to each other?

I think that the biggest obstacle here is establishing joint attention. It is easy to teach the names of visual stimuli to a seeing person. But when it comes to coining words to describe types of pain in the back, this becomes like two blind people trying to come up with words for categorizing shapes (they can experience shapes by touch, but without joint attention, i.e. let's say they are not allowed to pass shapes to each other).


Why are "the arts" traditionally visual and/or auditory? Because out of all our senses, vision and hearing are the only senses whose stimulus-response mapping is reliable enough. With the other senses, there is too much variation within and across people to have any control over their experience (which also explains why we have so few olfactory words/concepts). Smell and taste have very little spatiotemporal resolution. Touch may actually be a good candidate.

mirror of this post
8 comments   | comment & sign your name

hyper-abstracted R contest [02 Feb 2014|09:44pm]
(originally posted December 02, 2009, 21:44)

## `*` is the hyper of `+`, `^` is the hyper of `*`
> hyper <- function(fn) function(a,b) Reduce(fn, rep(a,b))

> compose <- function(fn1,fn2) function(x) fn1(fn2(x))

> hyperoperation <- function(n) Reduce(compose,listRep(hyper,n))(`+`)

('rep(obj,n)' and 'listRep(obj,n)' just return a list containing 'obj' n times. I had to invent 'listRep' for technical reasons, namely passing closures to 'rep' returns an error: "object of type 'closure' is not subsettable")

get it yet?Collapse )
14 comments   | comment & sign your name

my Burning Man checklist [18 Aug 2013|02:59am]
This is my first Burn... 8 days from now.

== desert weather ==
* camelbak: BOUGHT
* goggles: BOUGHT
* dust mask: BOUGHT

== camping ==
* tent: BORROWED
* rebar: BOUGHT
* reflective material for cooling: BOUGHT

== sleeping ==
* ear plugs (gel): BOUGHT
* sleep mask: ORDERED
* sleeping bag: ORDERED
* self-inflating mattress (3" thick): ORDERED
* blanket: ALREADY HAVE

== ride arrangements ==
I need to ride with Victoria, since we are splitting a Will Call ticket.
We arranged a ride in a sedan transporting 4 Burners and our stuff, which I think is very tight.
There might not be room for most of my stuff, so I will try to find other people who can transport my bicycle.

== lights ==
* headlamp: ORDERED
* reflective tape: ORDERED
* blinky reflective vest: ORDERED
* solar-powered light: BOUGHT
* bicycle lights: NEED BATTERIES

== electricity ==
* solar-powered phone charger: THANKS,GOOGLE
* batteries:

== survival ==
* dish, mug, cutlery: BOUGHT
* water:
* food:
* clothing for cold:

== MOOP ==
trash bags:

== other ==
duct tape: BOUGHT

mirror of this post
4 comments   | comment & sign your name

my taxes [10 Apr 2013|02:13pm]
I file taxes as a Resident Alien, which makes my tax situation pretty much identical to most American PhD students at Columbia. And yet, because of legal liability, the only qualified people who are willing to give us advice are tax professionals (whose time will cost at least $100).

So here's the basic calculations I do, before I start working on my tax forms. This is not tax advice. This is not legal advice.

Add up the income:
* TA Wages, see W2 form
* Stipend: look through bank account or MyColumbia, and add all the checks issued in 2012.
* Interest income: Chase sent me a form ("in lieu of 1099-INT"), informing me that I made $4.58 in interest, of which $0.00 was withheld. However, Chase charged an "Agent Admin Fee" $4.58, which means that I'm going to pay tax on money that I never saw... So let's be thankful that Savings accounts have such crappy interest!

Add up the withholdings:
* TA Wages, see W2 form
* Stipend: look through checks at MyColumbia. Check whether they withheld anything. In my case, they didn't.

Confusing things:
* My scholarship exactly cancels out tuition+fees. This means I don't need to look at the 1098-T, even though the university is obligated to send it to me. I think this form concerns the university's taxes wrt me, not my taxes.
* Unlike most international students, I should not receive a 1042-S (it's only for Non-Resident Aliens)

Since no money is being withheld from my stipend checks, I expect to owe money to the IRS, on the order of a few thousand per year.

My stipend is taxable

mirror of this post
3 comments   | comment & sign your name

summer sublet in NYC [29 Mar 2013|12:20pm]
I'm subletting my apartment. It's a great deal. You will not find this much space in such a nice area for $900/month. See here.

mirror of this post
1 comment   | comment & sign your name

conditional inference; why completeness matters [16 Nov 2012|01:00am]
Earlier this week, another piece of statistical theory fell into place for me, this time inspired by reading Cox&Hinkley.

One of the key principles expounded in this book is known as the "conditionality principle": given your model, if you can find a statistic that is ancillary (i.e. invariant to the parameter of interest), then your likelihood function should be conditional on it.

Now, if the minimal sufficient statistic is complete (as is the case in any full-rank exponential family), Basu's theorem tells us that any ancillary statistic will be independent of it, i.e. there is a clean separation between sufficient and ancillary. But in curved exponential families, it can happen that there is no maximal ancillary statistic, i.e. you may have multiple choices of ancillary statistic, but combining them yields a statistic that is no longer ancillary. This is a bit troubling to me, because it breaks the nice idea of a bijection between model and likelihood function.

Given a choice between two ancillaries, C&H advises selecting the one whose Conditional Fisher Information has the greater variance. It's not immediately obvious why one should do this, but I think this can be understood as the Conditional Fisher Information giving us a lens into the conditional likelihood function. For example, if the conditional Fisher Information has 0 variance, it may be because the ancillary statistic doesn't add any information (as is the case when the minimal sufficient statistic is complete). However, it still seems plausible to me that the Conditional Fisher Information can be constant (independent of the ancillary statistic) even while the likelihood function is sensitive to it.

C&H also hint at a notion of partial sufficiency/efficiency and how to measure it: just compute a Conditional Fisher Information, conditioning on the proposed statistic.

(Since Fisher Information is an expectation, Conditional Fisher Information is the expectation of a conditional distribution; since the quantity on the LHS is a function of the sufficient statistic, conditioning on the sufficient statistic will not change anything, whereas conditioning on something insufficient can have the effect of making the log-likelihood smoother, and the Fisher Information smaller) Conditioning on ancillary, however, doesn't simple make the log-likelihood sharper: the average of the Conditional Fisher Information is just the Fisher Information.

[the last paragraph is probably wrong; please comment]

mirror of this post
1 comment   | comment & sign your name

getting back on Twitter? [09 Nov 2012|12:45pm]
A lot of people in the Data Science world seem to be on Twitter. And I think it's true of techy people in general, at least the sort of techy people who are into Free Culture type things.

Of course, I much prefer having a real blog, with space to develop my thoughts and a permanent record. But there is something to be said for Twitter, which is hard to put my finger on. Twitter gives a feeling of immediacy, that you're speaking to the whole world. Although real blogs are just as public and just as immediate as Twitter, (a) blog interaction suffers from interoperability issues, especially commenting: there are many competing standards, and unfortunately OpenID didn't take off (though maybe Facebook Connect will), whereas Twitter is unified and simple; and (b) the set of people who read you somehow on real blogs feels more limited*. This may be because Twitter users often browse by hashtags (#), rather than users (@).

I have a little problem now: I have "followed" way too many people on Twitter, so I need to split my friends into groups, i.e. reading lists.

* - I, however, have had the opposite experience, which is part of why I abandoned Twitter the first time.

mirror of this post
2 comments   | comment & sign your name

the end of spam? [06 Nov 2012|04:25pm]
To my big surprise, the spam problem seems to be getting better, and it's not due to better spam filters or captchas, but rather to crackdowns on botnets. I do remember that it used to be worse.

Researcher: The End of Spam Is Closer Than You Think, July 2012

The End of Spam?, Jan 2011

mirror of this post
2 comments   | comment & sign your name

Greg Garing [22 Oct 2012|02:55am]
Tonight I saw the Drunken Master of country music, Greg Garing, at the Treehouse a.k.a. 2A, just the man and his guitar (thanks Kathryn Minogue). It was the craziest performance I've ever seen, quickly switching between mellow and ultra-hard rock; between quiet&romantic and rude. His vocal range is amazing, and I was especially entertained by his shivering bass. One sees that he is improvising the entire time, and cannot resist changing things around or cracking a joke (Running joke: "Can't you guys let me do one song properly from beginning to end?"). As far as ridiculous stunts go, I was especially impressed when he dropped his left arm and used the microphone stand as a slide for a whole blues turnaround, while looking totally out of it.

The event felt like a musicians' party full of old timers, people who had opened for Bob Dylan, and written reviews for Rolling Stone. The shows were projected live (in black&white) onto the red-brick building across the street, for a very nice effect. The bar staff were very chill, and told me that since they didn't serve food, I could bring outside food(!!!).

mirror of this post
comment & sign your name

intelligence augmentation and chess [15 Oct 2012|03:55pm]
Tyler Cowen's 8 insights on how human-computer collaboration in the context of chess
Insight #1, as one might imagine, is that human creativity is now worth more than it used to be, since most of the analysis is now automated.

mirror of this post
comment & sign your name

video storage [01 Oct 2012|09:06pm]
I've given up on compressing my videos. Most compression software doesn't compress batches of files, and those that do usually lose meta-data, like datetime. It also requires a ton of processing, which is slow and heats up my computer (this incidentally explains why my Canon digicams keep them in bulky formats, AVI or MOV: they are not nearly powerful enough to run compression).

So I've been moving my videos to my external G-Drive. But this 750GB drive, which also serves as a backup for my machine, is nearly full. This means that I have two problems:

* buy more space to store my videos.
* buy more space to have a backup of these videos.

I could buy 2x 2TB drives, but this is likely to be expensive.

Any suggestions?

mirror of this post
8 comments   | comment & sign your name

eye movements and cognitive state [26 Sep 2012|05:31pm]
Skeptical friends: some people say that when remembering things, your eyes move in a certain direction; when inventing things, they move in a different direction. The source of these ideas seems to be a pseudo-scientific field known as "Neuro-Linguistic Programming" (NLP). Is there any empirical basis for this claim?

See here

mirror of this post
2 comments   | comment & sign your name

yoga [10 Sep 2012|02:41am]
My yoga teacher, Joseph, is an atheist/skeptic, thinks very logically about what I need to work on. This seems to be very rare for yoga.

Today he told me that his "style" doesn't have a name, but that his teacher was Allan Bateman.

Finally, I asked him to name some materialistic schools/style of yoga. He told me:
* Krishnamurthi (empiricist philosophy FTW)
* Strala
* Katonah

(beware, the marketing may be mystical, but that's just marketing)

I might go to a Katonah lesson next week.


I seem to be making steady progress (e.g. I can now touch my toes after just a few minutes of stretching). But I'm still a long ways from where I want to be. There's a lot of work ahead for strenghtening my upper body (abdomen, chest, arms).

As of next week, I'm planning to do two lessons per week.

mirror of this post
2 comments   | comment & sign your name

causal inference [05 Sep 2012|10:22pm]
Although I love graphical models, I suspect that the "potential outcomes approach" is a more proper treatment of causal inference. See new book by Hernan and Robbins (h/t Michael Sobel).

Key concepts:
* counterfactuals: should be well-defined, but often aren't in the social science literature (e.g. to answer "what is the effect of marriage on health?", we'd have to imagine interventions that cause or prevent people from getting married; there are many non-equivalent ways to do this)
* potential outcomes: formalism in which all subjects are considered to have missing data for all but one experimental condition (i.e. the one that they were assigned to). This provides a direct way of thinking about token causation (a.k.a. causes-of-effects).
* ignorability: an important assumption that makes causal inference possible, similar to a missing-at-random assumption.
* propensity score matching: a way of coping when ignorability fails. (See also: Inverse probability weighting)

Directed graphical models perhaps provide something like a more concrete mechanism, allowing us to simulate the effects of interventions and propagate them downstream. But as far as real applications are concerned, papers in this tradition tend to make assumptions less explicit, and tend to mislead practitioners into thinking that the required assumptions are satisfied. (See Dawid - "Beware of the DAG")


UPDATE: Cosma Shalizi writes:
<< You've read Pearl's Statistics Surveys paper, right? I think the critique of the potential outcomes framework there, in section 4, is very strong. (Look at the stuff on ignorability, especially.) As for propensity matching, when the set of covariates you're using to calculate propensities doesn't meet the back door criterion, well, you get results like this. >>

mirror of this post
2 comments   | comment & sign your name

economic organization and taxation [04 Sep 2012|05:03pm]
Attention Conservation Notice: ranting about a topic that I know very little about; accuracy is sacrificed for the sake of cute analogies.

Taxation happens when you have transactions between separate entities. For example, when you buy/sell something, or pay/receive money for services. It makes no difference whether the tax is charged to the buyer or the seller, the employer or the employee.

Income tax on services encourages do-it-yourself and informal transactions: if you get your child to do it, no one is going to come into your house and audit your child. And the bigger your house, the more taxation you can avoid. Analogously, corporate income taxes encourage mergers & acquisitions: by bringing your supplier inside the organization, you no longer have to "buy" their product, since it is now made in-house! (Kinda like erecting an eruv). This may explain why corporations, unlike people, are taxed on profits; if supplies couldn't be deducted as business expenses, we would have double taxation and a huge effect on efficiency. (But surely double taxation is alive and well, no?)

It would seem natural to want to acquire your most important supplier. (Kinda like the signing up for a "best friend plan" on your mobile service). But are acquisitions just a simple way to dodge taxes? For one thing, now your big organization has to run a business that may be outside of its expertise, the newly-acquired business can end up insulated from market forces, and gradually lose its competitiveness, yadda yadda yadda. Now, this analysis conflates ownership with management. Of course, it is possible to buy the expertise required to run the sub-company (it will likely come with the package), but managers' interests won't necessarily align with company's. It is also possible to simulate a competitive environment (some large corporations implement competitive markets for supplies, machines, workers, and even venture capital).

The common justification for stopping mergers (and breaking up monopolies) is that they would make it impossible(?) to enforce rules against price-fixing.

I would like to see an empirical study of mergers. If we have a scenario in which a supplier has a single customer, is there any reason not to merge?


UPDATE: I've just convinced myself that corporate income taxes, if flat, have the nice property that they can't be gamed by mergers (Proof: the total profit after the merger will be the sum of the profits; this assumes that profits are positive). However, sales taxes can be avoided this way, for products that they use themselves (rather than resell).

Does a Major Company like Walmart have to Pay Sales Tax when Making Major Purchases From Another Business?
<< Businesses pay sales tax on items they purchase for their own use.
They don't pay sales tax on transactions in which they obtain products for resale in the store. This applies to all businesses of all sizes. Its also universal across state lines. >>

This means that merging along the production chain will not save on taxes. However, Walmart could save on taxes by buying a company that produces flooring or security cameras. Similarly, software companies could save on sales taxes by buying a coffee company.



* 5 tricks corporations use to avoid paying taxes

* When supply chains merge: 5 mistakes to avoid

* One Big Mutual Fund, or, The Ownership Society, by Cosma Shalizi
<< ... Ambitiously, Miller tries to explain why hierarchical corporations exist at all, why they take some of the forms they do, and how, in part, their form relates to their performance. ... >>

mirror of this post
6 comments   | comment & sign your name

DMV, 3rd attempt [21 Aug 2012|05:48pm]
Read more...Collapse )

mirror of this post
3 comments   | comment & sign your name

[ viewing | most recent entries ]
[ go | earlier ]