Kubke Lab:Research/ABR/Notebook/2013/11/05

From OpenWetWare

< Kubke Lab:Research | ABR | Notebook | 2013 | 11(Difference between revisions)
Jump to: navigation, search
(Andy)
Current revision (06:05, 5 November 2013) (view source)
(General Entries: Summary)
 
(6 intermediate revisions not shown.)
Line 11: Line 11:
=General Entries=
=General Entries=
-
* Insert content here.
+
* Skyped Andy vs Fabiana - trying to get an idea of the structure of the raw data files and make sure that the extracted text files are ok. Seems everything is ok, header log file contains all header info (albeit in a strange format). Different version files have different info, eg, some versions have the SD of the data others don't. Need to consider when extracting/analysing that the number of columns etc are going to vary from file to file.
=Personal Entries=
=Personal Entries=
==Fabiana==
==Fabiana==
 +
===Reading the binary files ===
*trying to open the actual .ABR files - getting nowhere. Noticed the log files that were extracted dont have the info I need and need to see what info is in the actual data files.  
*trying to open the actual .ABR files - getting nowhere. Noticed the log files that were extracted dont have the info I need and need to see what info is in the actual data files.  
Andy says it is an intel file - over skype trying to figure out the structure so we can read it - extracting info from  
Andy says it is an intel file - over skype trying to figure out the structure so we can read it - extracting info from  
Line 25: Line 26:
read next 1000 points,  
read next 1000 points,  
and so on
and so on
 +
 +
<pre>
 +
rawData = file("189L0A.ABR", "rb")
 +
a <- readBin(rawData,integer(),file.info("189L0A.ABR")$size,size = 2)
 +
#size2 says integer is 2 bytes
 +
close(rawData)
 +
plot(a)
 +
plot(a, ylim = c(-500,500))
 +
#at the begining there is 564points that correspond to the header
 +
#If I clean up the header
 +
plot(a[562:2565], type="line")
 +
str(a)
 +
b<-seq(1,length(a), 2)
 +
c<-seq(2,length(a),2)
 +
plot(a[b])
 +
plot(a[c])
 +
aclean<-a[562:2565]
 +
   
 +
head(aclean)
 +
head(aclean[b])
 +
head(aclean[c])
 +
plot(aclean[b], xlim=c(0,1000))
 +
plot(aclean[c], xlim=c(0,1000))
 +
 +
 +
</pre>
 +
 +
=== Extracting info from log file===
 +
# From file 2013-11-050b-MFK.Rmd (in sandbox)
 +
 +
<html>
 +
 +
<body>
 +
<h1>Trying to extract info from the log files</h1>
 +
 +
<p>Filename: &ldquo;419L76.ABR.log&rdquo;</p>
 +
 +
<p>This is what the notepad shows:</p>
 +
 +
<p>Owl  Date  Time    Clock_kHz  n_reps  freq    version
 +
419 05/30/93    14:22:22    50  25  5000    5.30
 +
    Comment line: Luxman -43 Auxkl B norm 100 valium used
 +
Channel 0 data, only</p>
 +
 +
<p>delay
 +
0
 +
-160
 +
-100
 +
-180
 +
-140</p>
 +
 +
<p>I want to extract
 +
[1:7] -&gt; headers
 +
[8:14] -&gt; info
 +
rest -&gt; comment line
 +
tail -&gt; delays</p>
 +
 +
<pre><code class="r">logfile &lt;- read.delim2(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, header = T)
 +
head(logfile)
 +
</code></pre>
 +
 +
<pre><code>##                    Owl
 +
## 1                  419
 +
## 2                   
 +
## 3 Channel 0 data, only
 +
## 4                delay
 +
## 5                    0
 +
## 6                -160
 +
##                                                    Date    Time Clock_kHz
 +
## 1                                              05/30/93 14:22:22        50
 +
## 2 Comment line: Luxman -43 Auxkl B norm 100 valium used                NA
 +
## 3                                                                      NA
 +
## 4                                                                      NA
 +
## 5                                                                      NA
 +
## 6                                                                      NA
 +
##  n_reps freq version
 +
## 1    25 5000    5.30
 +
## 2    NA  NA       
 +
## 3    NA  NA       
 +
## 4    NA  NA       
 +
## 5    NA  NA       
 +
## 6    NA  NA
 +
</code></pre>
 +
 +
<pre><code class="r">str(logfile)
 +
</code></pre>
 +
 +
<pre><code>## &#39;data.frame&#39;:    9 obs. of  7 variables:
 +
##  $ Owl      : Factor w/ 9 levels &quot;&quot;,&quot;-100&quot;,&quot;-140&quot;,..: 7 1 8 9 6 4 2 5 3
 +
##  $ Date    : Factor w/ 3 levels &quot;&quot;,&quot;05/30/93&quot;,..: 2 3 1 1 1 1 1 1 1
 +
##  $ Time    : Factor w/ 2 levels &quot;&quot;,&quot;14:22:22&quot;: 2 1 1 1 1 1 1 1 1
 +
##  $ Clock_kHz: int  50 NA NA NA NA NA NA NA NA
 +
##  $ n_reps  : int  25 NA NA NA NA NA NA NA NA
 +
##  $ freq    : int  5000 NA NA NA NA NA NA NA NA
 +
##  $ version  : Factor w/ 2 levels &quot;&quot;,&quot; 5.30&quot;: 2 1 1 1 1 1 1 1 1
 +
</code></pre>
 +
 +
<p>I get a funny looking table</p>
 +
 +
<pre><code class="r">loghead &lt;- read.table(&quot;419L76.ABR.log&quot;, header = T, sep = &quot;\t&quot;, nrows = 1, skip = 0)
 +
</code></pre>
 +
 +
<p>This gets me the descriptor - still need to get the other rows</p>
 +
 +
<pre><code class="r">loghead &lt;- read.table(&quot;419L76.ABR.log&quot;, header = T, sep = &quot;\t&quot;, nrows = 1, skip = 0)
 +
loganaesth &lt;- read.table(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, nrows = 1, skip = 2)
 +
logtail &lt;- read.table(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, col.names = &quot;Owl&quot;, nrows = 20,
 +
    skip = 3)
 +
print(loghead)
 +
</code></pre>
 +
 +
<pre><code>##  Owl    Date    Time Clock_kHz n_reps freq version
 +
## 1 419 05/30/93 14:22:22        50    25 5000    5.3
 +
</code></pre>
 +
 +
<pre><code class="r">print(loganaesth)
 +
</code></pre>
 +
 +
<pre><code>##  V1                                                    V2
 +
## 1 NA Comment line: Luxman -43 Auxkl B norm 100 valium used
 +
</code></pre>
 +
 +
<pre><code class="r">print(logtail)
 +
</code></pre>
 +
 +
<pre><code>##                    Owl
 +
## 1 Channel 0 data, only
 +
## 2                delay
 +
## 3                    0
 +
## 4                -160
 +
## 5                -100
 +
## 6                -180
 +
## 7                -140
 +
</code></pre>
 +
 +
<p>Data file 419L76.ABR.txt
 +
read.table gives me an error that there is a mismatch betweeen columns and column names - but read.delim2 seems to solve the problem, although it creates a last column x with NAs (which is probably what is generating the problem in hte first place.)</p>
 +
 +
<pre><code class="r">data &lt;- read.delim2(&quot;419L76.ABR.txt&quot;, sep = &quot;\t&quot;, header = T)
 +
head(data)
 +
</code></pre>
 +
 +
<pre><code>##      msec left right spont bin.0 bin.1 bin.2 bin.3 bin.4 sdev_l sdev_r
 +
## 1    0.00  -93  -39    37  -49  -23    -1  -25  -140    101    107
 +
## 2    0.02    0  -49    20  -67  -12    86  158  -146    187    104
 +
## 3    0.04  -69  -70    21  -75  -23    24    3  -149    88    100
 +
## 4    0.06  -71  -107    35  -75  -38    35    4  -120    89    99
 +
## 5    0.08  -67  -117    16  -67  -45    41    8  -84    94    107
 +
## 6    0.10  -48  -98  -22  -46  -49    34    9  -71    95    103
 +
##  sdev_spont sd_bin.0 sd_bin.1 sd_bin.2 sd_bin.3 sd_bin.4  X
 +
## 1        112      86      101      129      126      116 NA
 +
## 2        112      101      96      195      260      118 NA
 +
## 3        109      113      100      123      128      110 NA
 +
## 4        106      108      106      130      119      99 NA
 +
## 5        99      95      106      140      117      99 NA
 +
## 6        95      91      109      139      115      104 NA
 +
</code></pre>
 +
 +
<p>returns 5 columns of bin.[0:4] - which probably correspond to the 5 delays in the log file. </p>
 +
 +
</body>
 +
 +
</html>
==Andy==
==Andy==
 +
 +
===ABR.H header file===
 +
Following file is from the ABR.H header file that was used for the original files.
Following file is from the ABR.H header file that was used for the original files.

Current revision

Hearing development in barn owls Main project page
Previous entry      Next entry

Contents


General Entries

  • Skyped Andy vs Fabiana - trying to get an idea of the structure of the raw data files and make sure that the extracted text files are ok. Seems everything is ok, header log file contains all header info (albeit in a strange format). Different version files have different info, eg, some versions have the SD of the data others don't. Need to consider when extracting/analysing that the number of columns etc are going to vary from file to file.

Personal Entries

Fabiana

Reading the binary files

  • trying to open the actual .ABR files - getting nowhere. Noticed the log files that were extracted dont have the info I need and need to see what info is in the actual data files.

Andy says it is an intel file - over skype trying to figure out the structure so we can read it - extracting info from abr.h and abr_4_mac.c

According to andy need to Read the first 1128 bytes then read the next 1000 2-byte int skip 52 read next 1000 points, and so on

rawData = file("189L0A.ABR", "rb")
a <- readBin(rawData,integer(),file.info("189L0A.ABR")$size,size = 2) 
#size2 says integer is 2 bytes
close(rawData)
plot(a) 
plot(a, ylim = c(-500,500))
#at the begining there is 564points that correspond to the header
#If I clean up the header 
plot(a[562:2565], type="line")
str(a)
b<-seq(1,length(a), 2)
c<-seq(2,length(a),2)
plot(a[b]) 
plot(a[c])
aclean<-a[562:2565]
     
head(aclean)
head(aclean[b])
head(aclean[c])
plot(aclean[b], xlim=c(0,1000))
plot(aclean[c], xlim=c(0,1000))


Extracting info from log file

  1. From file 2013-11-050b-MFK.Rmd (in sandbox)

Trying to extract info from the log files

Filename: “419L76.ABR.log”

This is what the notepad shows:

Owl Date Time Clock_kHz n_reps freq version 419 05/30/93 14:22:22 50 25 5000 5.30 Comment line: Luxman -43 Auxkl B norm 100 valium used Channel 0 data, only

delay 0 -160 -100 -180 -140

I want to extract [1:7] -> headers [8:14] -> info rest -> comment line tail -> delays

logfile <- read.delim2("419L76.ABR.log", sep = "\t", header = T)
head(logfile)
##                    Owl
## 1                  419
## 2                     
## 3 Channel 0 data, only
## 4                delay
## 5                    0
## 6                 -160
##                                                    Date     Time Clock_kHz
## 1                                              05/30/93 14:22:22        50
## 2 Comment line: Luxman -43 Auxkl B norm 100 valium used                 NA
## 3                                                                       NA
## 4                                                                       NA
## 5                                                                       NA
## 6                                                                       NA
##   n_reps freq version
## 1     25 5000    5.30
## 2     NA   NA        
## 3     NA   NA        
## 4     NA   NA        
## 5     NA   NA        
## 6     NA   NA
str(logfile)
## 'data.frame':    9 obs. of  7 variables:
##  $ Owl      : Factor w/ 9 levels "","-100","-140",..: 7 1 8 9 6 4 2 5 3
##  $ Date     : Factor w/ 3 levels "","05/30/93",..: 2 3 1 1 1 1 1 1 1
##  $ Time     : Factor w/ 2 levels "","14:22:22": 2 1 1 1 1 1 1 1 1
##  $ Clock_kHz: int  50 NA NA NA NA NA NA NA NA
##  $ n_reps   : int  25 NA NA NA NA NA NA NA NA
##  $ freq     : int  5000 NA NA NA NA NA NA NA NA
##  $ version  : Factor w/ 2 levels ""," 5.30": 2 1 1 1 1 1 1 1 1

I get a funny looking table

loghead <- read.table("419L76.ABR.log", header = T, sep = "\t", nrows = 1, skip = 0)

This gets me the descriptor - still need to get the other rows

loghead <- read.table("419L76.ABR.log", header = T, sep = "\t", nrows = 1, skip = 0)
loganaesth <- read.table("419L76.ABR.log", sep = "\t", nrows = 1, skip = 2)
logtail <- read.table("419L76.ABR.log", sep = "\t", col.names = "Owl", nrows = 20, 
    skip = 3)
print(loghead)
##   Owl     Date     Time Clock_kHz n_reps freq version
## 1 419 05/30/93 14:22:22        50     25 5000     5.3
print(loganaesth)
##   V1                                                    V2
## 1 NA Comment line: Luxman -43 Auxkl B norm 100 valium used
print(logtail)
##                    Owl
## 1 Channel 0 data, only
## 2                delay
## 3                    0
## 4                 -160
## 5                 -100
## 6                 -180
## 7                 -140

Data file 419L76.ABR.txt read.table gives me an error that there is a mismatch betweeen columns and column names - but read.delim2 seems to solve the problem, although it creates a last column x with NAs (which is probably what is generating the problem in hte first place.)

data <- read.delim2("419L76.ABR.txt", sep = "\t", header = T)
head(data)
##       msec left right spont bin.0 bin.1 bin.2 bin.3 bin.4 sdev_l sdev_r
## 1     0.00  -93   -39    37   -49   -23    -1   -25  -140    101    107
## 2     0.02    0   -49    20   -67   -12    86   158  -146    187    104
## 3     0.04  -69   -70    21   -75   -23    24     3  -149     88    100
## 4     0.06  -71  -107    35   -75   -38    35     4  -120     89     99
## 5     0.08  -67  -117    16   -67   -45    41     8   -84     94    107
## 6     0.10  -48   -98   -22   -46   -49    34     9   -71     95    103
##   sdev_spont sd_bin.0 sd_bin.1 sd_bin.2 sd_bin.3 sd_bin.4  X
## 1        112       86      101      129      126      116 NA
## 2        112      101       96      195      260      118 NA
## 3        109      113      100      123      128      110 NA
## 4        106      108      106      130      119       99 NA
## 5         99       95      106      140      117       99 NA
## 6         95       91      109      139      115      104 NA

returns 5 columns of bin.[0:4] - which probably correspond to the 5 delays in the log file.

Andy

ABR.H header file

Following file is from the ABR.H header file that was used for the original files.

File layout is probably: <Master_Header><Sub_Structure><1000 2-byte INT><Sub_Structure><1000 2-byte INT> ...

This structure needs to be confirmed.



typedef struct

   {
   short freq;	/* frequency in Hz */
   short amp;	/* amplitude in D/A units */
   short dur;		/* duration in clock ticks */
   short rise_fall;	/* rise_fall in clock ticks */
   } SIGNAL; // 8 bytes

typedef struct

   {
   SIGNAL carrier;
   SIGNAL envelope;
   } STIMULUS;

typedef struct

   {
   short left_delay;	/* microseconds to left  onset */
   short right_delay;	/* microseconds to right onset */
   short rep_shorterval;	/* interval in millseconds */
   } TIMING; //6 bytes

typedef struct

   {
   short clock;	    /* clock frequency in kHz */
   short max_size;   /* maximum size of each array */
   short channels;   /* 1 for carrier only, 2 for carrier & envelope */
   } DA_INFO; //6 bytes

typedef struct

   {
   short clock;	    /* in kHz */
   short size;	    /* number of elements (per channel) */
   short channels;   /* number of channels */
   short avg_reps;   /* number of reps making up the average */
   } AD_INFO; //8 bytes

typedef struct

   {
   char date[12];
   char time[12];
   char owl_id[12];
   char exp_id[12];
   char comments[80];
   short depth;
   short version;
   char side;
   short num_chan;
   short num_tests;
   short gain;
   char extra_room[1024-(12+12+12+12+80+2+2+1+2+2+2)];/* Add two more to correct CHAR */
   //    char extra_room[1024-(12+12+12+12+80+2+2+1)];/* Add two more to correct CHAR */
   } HEADER_INFO,  EXPERIMENT_HEADER; // Bytes 1024

typedef struct /**** THE MASTER STRUCTURE ****/

   {
   HEADER_INFO header; //1024
   STIMULUS left;      //  16
   STIMULUS right;     //  16
   TIMING delay;       //   6
   DA_INFO output;     //   6
   AD_INFO input;      //   8
   } EXPERIMENT;   // 1076 bytes


typedef struct /**** SUB STRUCTURE ****/

   {
   STIMULUS left;      //    16
   STIMULUS right;     //    16
   TIMING delay;       //     6
   DA_INFO output;     //     6
   AD_INFO input;      //     8
   } EXPERIMENT_INFO; // 52 bytes

Oris

  • Enter content here


Personal tools