User:Timothee Flutre/Notebook/Postdoc/2012/09/12 - Revision history

Timothee Flutre: /* Handling compressed files with gzip in C++ */ rmv redundant spaces in code

Timothee Flutre — Thu, 13 Sep 2012 20:04:26 GMT

Handling compressed files with gzip in C++: rmv redundant spaces in code

←Older revision		Revision as of 20:04, 13 September 2012
Line 17:		Line 17:

	<nowiki>		<nowiki>
-	// Author: Timothee Flutre	+	// Author: Timothee Flutre
-	// Aim: show how to use Zlib in your own C++ code	+	// Aim: show how to use Zlib in your own C++ code
-	// Compilation: g++ -Wall -Wextra -g test.cpp -lz	+	// Compilation: g++ -Wall -Wextra -g test.cpp -lz
-	// Not copyrighted -- provided to the public domain	+	// Not copyrighted -- provided to the public domain

	#include <cmath>		#include <cmath>
Line 34:		Line 34:
	#include "zlib.h"		#include "zlib.h"

-	//-----------------------------------------------------------------------------	+	//-----------------------------------------------------------------------------
-	// copy-paste and use the four following functions in your own code	+	// copy-paste and use the four following functions in your own code

	void		void
Line 77:		Line 77:
	{		{
	c = gzgetc (fileStream);		c = gzgetc (fileStream);
-	if (c == -1) // eof or error	+	if (c == -1) // eof or error
	{		{
	res = 0;		res = 0;
	break;		break;
	}		}
-	else if (c == 10) // 10 is ASCII code for '\n'	+	else if (c == 10) // 10 is ASCII code for '\n'
	break;		break;
	else		else
Line 97:		Line 97:
	const size_t & lineId)		const size_t & lineId)
	{		{
-	// if (gzprintf (fileStream, "%s", line.c_str()) <= 0)	+	// if (gzprintf (fileStream, "%s", line.c_str()) <= 0)
	if (gzputs (fileStream, line.c_str()) < 0)		if (gzputs (fileStream, line.c_str()) < 0)
	{		{
Line 106:		Line 106:
	}		}

-	//-----------------------------------------------------------------------------	+	//-----------------------------------------------------------------------------
-	// functions to show how it works	+	// functions to show how it works

	void		void
Line 114:		Line 114:
	cout << __FUNCTION__ << endl;		cout << __FUNCTION__ << endl;

-	// create a dummy file via a system call	+	// create a dummy file via a system call
	system ("rm -f data.txt.gz; echo -e \"aaa\nbb\nccc\" \| gzip > data.txt.gz");		system ("rm -f data.txt.gz; echo -e \"aaa\nbb\nccc\" \| gzip > data.txt.gz");

-	// read the dummy file using zlib API	+	// read the dummy file using zlib API
	string fileName = "data.txt.gz";		string fileName = "data.txt.gz";
	gzFile fileStream;		gzFile fileStream;
-	openFile (fileName, fileStream, "rb"); // works also if file not compressed	+	openFile (fileName, fileStream, "rb"); // works also if file not compressed
	string line;		string line;
	size_t lineId = 0;		size_t lineId = 0;
Line 136:		Line 136:
	closeFile (fileName, fileStream);		closeFile (fileName, fileStream);

-	// clean	+	//clean
	system ("rm -f data.txt.gz");		system ("rm -f data.txt.gz");
	}		}
Line 144:		Line 144:
	cout << __FUNCTION__ << endl;		cout << __FUNCTION__ << endl;

-	// write a dummy file using zlib API	+	// write a dummy file using zlib API
	string fileName = "data.txt.gz";		string fileName = "data.txt.gz";
	gzFile fileStream;		gzFile fileStream;
Line 156:		Line 156:
	closeFile (fileName, fileStream);		closeFile (fileName, fileStream);

-	// read it via a system call	+	// read it via a system call
	system ("zcat data.txt.gz");		system ("zcat data.txt.gz");

-	// clean	+	// clean
	system ("rm -f data.txt.gz");		system ("rm -f data.txt.gz");
	}		}

Timothee Flutre: /* Handling compressed files with gzip in C++ */ add code

Timothee Flutre — Thu, 13 Sep 2012 19:58:29 GMT

Handling compressed files with gzip in C++: add code

Show changes

Timothee Flutre: /* Entry title */ first version (without code)

Timothee Flutre — Wed, 12 Sep 2012 21:47:27 GMT

Entry title: first version (without code)

←Older revision		Revision as of 21:47, 12 September 2012
Line 6:		Line 6:
	\| colspan="2"\|		\| colspan="2"\|
	<!-- ##### DO NOT edit above this line unless you know what you are doing. ##### -->		<!-- ##### DO NOT edit above this line unless you know what you are doing. ##### -->
-	==~~Entry title~~==	+	==Handling compressed files with gzip in C++==
-	* ~~Insert content here~~...	+
		+	* It's more and more common in biology to handle large amount of data, and thus more and more required to work with compressed files. The [http://www.gzip.org/ gzip] programs offers a good balance between compression speed and size. That's why high-level languages such as [http://docs.python.org/library/gzip.html Python] and [http://stat.ethz.ch/R-manual/R-patched/library/base/html/connections.html R] natively provide ways to handle files compressed with gzip. But what about C++?
		+
		+	* The [http://www.cs.unc.edu/Research/compgeom/gzstream/ gzstream] library is often mentioned as a good solution. However, it's likely to be already installed neither on your machine, nor the one of people interested in your code. So you and them will have to install it (or you'll have to distribute it with your own package). Moreover, it doesn't support seek and is unlikely to do so in the near future. Also, if you want your code to successfully read files whether they are compressed or not, you will have to check the extension of the file name by yourself (.gz) and use ifstream if uncompressed and igzstream otherwise: messy...
		+
		+	* Why not using [http://www.zlib.net/ zlib] directly? If you work on Linux it's already installed (it's used by the Linux kernel), and if you work on Mac OS it's likely to be already there also. (And it even works on Windows, but who cares?) More importantly, if you're not a professional software developer, it happens to be pretty easy to use zlib. Below is an example code showing how I typically use it in my own C++ code..

Timothee Flutre: Autocreate 2012/09/12 Entry for User:Timothee_Flutre/Notebook/Postdoc

Timothee Flutre — Wed, 12 Sep 2012 21:31:55 GMT

Autocreate 2012/09/12 Entry for User:Timothee_Flutre/Notebook/Postdoc

New page

{|{{table}} width="800"
|-
|style="background-color: #EEE"|[[Image:owwnotebook_icon.png|128px]]<span style="font-size:22px;"> Project name</span>
|style="background-color: #F2F2F2" align="center"|<html><img src="/images/9/94/Report.png" border="0" /></html> [[{{#sub:{{FULLPAGENAME}}|0|-11}}|Main project page]]<br />{{#if:{{#lnpreventry:{{FULLPAGENAME}}}}|<html><img src="/images/c/c3/Resultset_previous.png" border="0" /></html>[[{{#lnpreventry:{{FULLPAGENAME}}}}{{!}}Previous entry]]<html>      </html>}}{{#if:{{#lnnextentry:{{FULLPAGENAME}}}}|[[{{#lnnextentry:{{FULLPAGENAME}}}}{{!}}Next entry]]<html><img src="/images/5/5c/Resultset_next.png" border="0" /></html>}}
|-
| colspan="2"|

==Entry title==
* Insert content here...


|}

__NOTOC__