DataONE:Notebook/Summer 2010/2010/07/19

Heather: Hi Nic!
 
me: Hi Heather
 
Heather: Now a good time for talking R?
 
me: sure
 
Heather: kudos btw on getting the submission off on Friday
9:59 AM 
me: do you want to use the desktop sharing application I have or just do it via chat?
 
Heather: maybe chat, and then if we do desktop sharing maybe I'll try Adobe Connect
 
me: oh no thank you and todd for your input
  
ok
 
Heather: a few other projects I am on use that, so probably would behoove me to get more familiar with it
10:00 AM 
you have R on the computer where you are?
 
me: we have this suite of GoTo for my program.. it's pretty awesome
  
very easy to use, but I think it's pricey
 
Heather: do you a dataset you'd like to work with first?
 
me: yes
 
Heather: good to know. yeah, mostly pricey isn't my thing these days :)
 
me: I got the dataset into R using the command you gave me for datraw
 
Heather: online? want to point me to it?
  
great
10:01 AM 
let me pull that up
  
can you run all the example commands sucessfully, does it look like?
 
me: I couldn't get the graphs to appear
 
Heather: hrm ok
  
are you on a mac?
 
me: when I tried to plot something nothing showed up
  
yes
10:02 AM 
should I send you the .csv?
 
Heather: running the normal R program (as opposed to some variant like JGR or sometihng)?
  
nah, I got it.
 
me: R version 2.11.1
 
Heather: does it complain about not plotting, or it just silently does nothing?
10:03 AM 
me: Im in the console
  
oh wow
 
Heather: I'm version 2.10.1
  
shouldn't matter much though
 
me: actually it just magically worked
 
Heather: oh wow?
  
oh good
  
the magic of someone else watching, eh?
 
me: ha
10:04 AM 
Heather: great. ok, do the plots and just make sure they all seem to work....
10:06 AM 
me: the plots seem to work
 
Heather: great.
 
me: the abline did not
 
Heather: and they make a bit of sense?
  
or do you want to ask what they are about?
  
hrm, don't know why not
 
me: I think these make a bit of sense
 
Heather: oh well, that isn't very relevant for now, abline
 
me: abline is ?
10:07 AM 
Heather: yeah, it plots a best fit line
  
on top of the scatter plot
  
mostly it was just showing off some of what could be done
 
me: I've been reading a begginers guide to R this morning, but not up to speed obviously
 
Heather: and giving a gut feel for regression
 
me: ok
 
Heather: so let's keep going then
  
if there is a problem with abline it will probably show up in something else too
  
so we'll just deal with it then.
10:08 AM 
ok dokee. do you have thoughts about what stats you think you want to do?
  
or, put another way, do you have ideas about hypotheses you want to test or expore with numbers?
10:09 AM 
me: well, to be honest, I don't... I would like to know most about the journals what kind of correlations we can make between requested or required sharing and impact metrics
 
Heather: ok.
  
in that case, I think let's start with the numbers you pulled together for your poster
  
since those were obviously of interest :)
 
me: ok
 
Heather: and we'll move on from there
  
(this morning, or otherwise soon)
  
so... you calculated some percentages
10:10 AM 
percentages are useful, but they lack information by themselves
  
they don't portray how confident that we should be in the fact that the percentages would generalize, right?
  
so we suppliment them with an n= number
 
me: right
 
Heather: to sort of show how big the sample was that we used to calc the percentages
10:11 AM 
obviously if n=100000 we thing the percentages might be more valid/precise/reproducible/etc than if n=10
10:12 AM 
a different way to think about that is that if n=100000, it is really unlikely that the percentage was actually radically different than what we measured
  
and we just got what we measured by chance
  
whereas if n=10, it is totally possible that the mean is really different and we just got what we got by chance
10:13 AM 
one way to use the n and the percentage to encapsulate this idea
  
is to calculate confidence intervals around the percentages
  
have you heard of them? do you understand them, a lot , a little, not really at all?
10:14 AM 
me: heard of them, do not know what they are though
 
Heather: ok, no prob
  
hrm, do you have one of your percentages from your poster handy?
 
me: sure one sec
 
Heather: let me pull up your poster draft, and if you could do that too it woudl be great
10:15 AM 
me: FINDINGS
Funding Agencies (53 total evaluated): 44% (23 of 53) require data sharing in some way; 8% (4) specify the duration for which data must be preserved by the primary investigator; 25% (13) give directions on the type of repository to which data should be deposited (while only 2 name particular repositories); 8% (4) provide supplemental funds available for deposition of data; only one gives direction on how such data is to be cited.
 
Heather: right. great. or for journals, the dataset we have in R right now, "13% (40 of 307) request or require data to be archived;"
  
so 13% is useful, but we want more info.
  
we want to know, might it be 12%? 14%?
  
what about 5%? 25%
10:16 AM 
how confident are we that the actual percentage, if we were able to measure the entire sample that this came from, would be 13%?
  
so one way we do this is to calculate "confidence intervals"
  
the most common kind are 95% confidence intervals.
10:17 AM 
if the confidence intervals were, say 8% to 17%, then that would mean
  
that given the data we have,
  
our best guess is that if we were to repeat this experiement 100 times
10:18 AM 
only 5% of the time would we estimate the percentage to be lower than 8% or higher than 17%
  
just by chance/bad luck
  
this idea takes a while to sink in, so maybe I'll try to say it a few ways.
10:19 AM 
we are 95% sure, given the sample size we used and the data we saw
  
that the real percentage value in the whole universe that we sampled from is between 8 and 17%
  
it is possible it is lower than that or higher than that
  
in fact it could be anything
  
but if it were actually lower than that or higher than that
10:20 AM 
it woudl make the sampel that we got really surprising
  
and extreme surprises don't happen very often
 
me: ok this makes sense
 
Heather: so our best guess is that the real percentage is close to what we got
  
great!
  
given the same percentage,
10:21 AM 
you'll find that estimates made with small sample sizes end up with wider confidence intervals
 
me: so just to make sure, is 95% arbitrary or is that a standard?
 
Heather: ie 3 ti 25% or something
  
whereas large sample sizes end up with tight confidence intervals: 10 to 15% in this example
  
95% is standard
  
it ties directly to the p<0.05 threshold that is also standard,
10:22 AM 
that you'lve likely heard about
 
me: ok
 
Heather: they are mostly two ways of measuring the same thing
  
the amount of surprise/bad luck etc you'd have to get to have to get a sample that was so strange/disproportionate to have given you a weird estimate, if the real actual value was something different
10:23 AM 
the tightness of the confidence intervals also depends on what the percentage is.
  
but no need to go into that now
  
for now let me just show you how to caluclate them to you can explore :)
  
if you do ?binconf in R, what happens?
 
me: ok
10:24 AM 
No documentation for 'biconf' in specified packages and libraries:
you could try '??biconf'
 
Heather: ok.
  
if you do "library(Hmisc)" what happens?
10:25 AM 
me: no package called hmisc
 
Heather: ok. do you have a "packages & data" menu at the top?
  
go to package installer in that menu
 
me: ok
10:26 AM 
Heather: then with CRAN (binaries) selected
  
search for Hmisc
  
(caps matter)
  
then "Get List)
 
me: 3.8-2?
 
Heather: do you see an Hmisc package?
  
ypu
 
me: I mean does the version matter?
 
Heather: highlight it, check "install dependencies", and click "install selected"
  
no version doesn't matter much now
10:27 AM 
sometimes it can, but I don't think it will for what we are doing
  
does it look like it is installing?
 
me: its done
 
Heather: great
  
then type library(Hmisc)
  
then ?binconf
10:28 AM 
me: ok
  
its giving me a table
  
with topic and package and description
 
Heather: really? hrm, at the prompt?
10:29 AM 
well, try instead at the prompt typing
  
binconf(40, 307)
 
me: cant find function biconf
10:30 AM 
Heather: ok. when at the prompt you type
  
library(Hmisc)
  
what happens?
 
me: I get another prompt
 
Heather: good
  
then when you type
  
?binconf
  
what happens?
10:31 AM 
me: I get a pop up window that gives me descriptions of conf intervals for binomial probabilities
 
Heather: great
  
now at the prompt type
  
binconf(40, 307)
 
me: PointEst Lower Upper
0.1302932 0.0971623 0.1725619
 
Heather: yay
  
so that there are your confidence intervals
10:32 AM 
me: sorry I am bit slow
 
Heather: no, that's fine
 
me: must have mistyped something before
 
Heather: easy to make typos etc in this way of doing it
  
no prob
 
me: so lower upper look to be what you said right?
 
Heather: so if you look at the popup from ?binconf
 
me: 9 and 17
 
Heather: you can see that alpha=0.05
  
the confidence is 1-alpha, so these are 95% confidence intervals
10:33 AM 
if you wanted to computer 99% confidence intervals
  
(which would be wider, right, because to be really sure of something
10:34 AM 
you need to include at least what you thought before, plus give more wiggle room for even stranger subsamples)
  
you would enter
  
binconf(40, 307, 0.01)
 
me: PointEst Lower Upper
0.1302932 0.0885323 0.1876962
 
Heather: but really you only care about 95% confidence intervals, so you can forget that part :)
  
yup
10:35 AM 
so you could write a text file that computes the confidence intervals for all of the percentages in your poster, for example
  
that would convey information that is useful for others reading it
 
me: ok
 
Heather: they could know, ok, it is pretty unlikely that the percentage is actually as low as 5%, given this evidence.....
  
etc
10:36 AM 
so that is the first bit of useful stats you can do
  
if you were to write all of those binconf lines, for example
  
you could save them in a text file
  
and give it the extension .R
  
and then it would be called an R script
  
and you could run it either by copying and pasting the contents into the prompt
  
or by typing
10:37 AM 
source(yourfilename.R)
  
at the command line
  
(or some other ways from the menus, I think)
  
it is useful to make and save these .R files
  
(especially with loading the dataset at the top)
10:38 AM 
because they document the stats that you ran
  
and make it very easy and transparent for other people to see what you did
  
make sense?
 
me: um, so when you say write out all the binconf, you mean write them in a text editor like emacs right and then load it into R
10:39 AM 
Heather: yes
 
me: if I save it as an r script
  
ok
 
Heather: yes
  
basically you'd have a lot of lines of bioconf, one for each percentage you calculate
 
me: so does r allow you to use # for comments
 
Heather: maybe a varient of this to make it easier to read
  
round(100*binconf(40, 307), 1)
  
yes
10:40 AM 
me: that way I could name them
  
ok good
 
Heather: I'm not necessarily suggesting that you do this
  
there are some better ways to do it more easily, perhaps,
  
but it might be worth doing it for a few to get the hang of it
  
and a gut feel for how "wide" your confidence intervals are for your different data sets,
10:41 AM 
given the questions that you asked and the differeing sizes of your samples
 
me: ok so I should just explore a bit with these
 
Heather: yup
 
me: ok
10:42 AM 
Heather: let me give you a bit more to chew on before I run to my next eeting
  
try this
  
Hmisc::describe(dat.raw)
  
you get a long list of stuff?
 
me: yup
10:43 AM 
Heather: ok, that can be a good way to have a look at your data
 
me: whoa this is awesome
 
Heather: one of the things you will realize as you do that
  
is that some of your fields aren't coded for easy stat analysis yet
  
for example,
10:44 AM 
dat.raw$Policy.Requested.or.Required.Authors.to.share.data..for.publication
  
has some random comments in it
  
which is useful, but you need a cleaner column that that if you want to look at a percentage, right?
10:45 AM 
one way you can get a cleaner column is to make one automatically.
 
me: right
 
Heather: for example, you could make a new column called "requested"
  
that is either TRUE or FALSE
 
me: so I tried to code these doing 0,1,2
 
Heather: ok.
  
let me show you one way do do it useing text matches, in case that is helpful for this or somehting else....
 
me: creating a new column that said 0 no request 1 request 2 required ( for example)
10:46 AM 
Heather: two steps
  
first make a new column that is all false
  
dat.raw$requested = FALSE
  
then, change just some of them to true, the ones that contain the word "Requested"
  
dat.raw$requested[grep("+Requested+", dat.raw$Policy.Requested.or.Required.Authors.to.share.data..for.publication)] = TRUE
10:47 AM 
then when you type dat.raw$requested
  
you can see the cleaner column
  
this may not be ideal for this case, I agree a manual coding of 0,1,2 might be better
  
but maybe for pulling out various publishers, ISI categories, whatever
  
one more thing
  
once you have a binary column
10:48 AM 
= one that is just TRUE/FALSE or 1/0
  
you can do a nice plot to look at the odds of something
  
so the percentage time that it happens, in a subset of your data
10:49 AM 
can you run the two dat.raw$requested lines above, first?
  
so
  
dat.raw$requested = FALSE
  
then
  
dat.raw$requested[grep("+Requested+", dat.raw$Policy.Requested.or.Required.Authors.to.share.data..for.publication)] = TRUE
 
me: I got this response-- Error in `$<-.data.frame`(`*tmp*`, "requested", value = c(NA, NA, TRUE, : 
replacement has 34 rows, data has 39
10:50 AM 
I think I need to clean up the .csv file ?
 
Heather: Hmmmm I wonder if I cleaned up my data a bit more than you did
  
yeah, I'm guessing so
 
me: ok
10:52 AM 
Heather: hrm, I was trying to show you something else but out of time to get it to work
  
I have a 3 hr online meeting now
 
me: ick
 
Heather: then could be avail from 2-3 my time
  
or after 7 my time
  
or tomorrow.....
 
me: whatever works best for you
  
I am going to make sure these are more normalized
 
Heather: ok, let's play it by ear a bit
10:53 AM 
me: and then play around with what you gave me
 
Heather: yup, normalizing would help
 
me: I should be online most of the day though
 
Heather: great
 
me: I;ll leave my chat open
 
Heather: send me email if/when you run into things that are confusing?
 
me: ok
 
Heather: ok.
  
looking forward to getting into more good stuff today and tomorrow :)
  
bye!
10:54 AM 
me: thanks for the help!
  
bye