decompression problem

[Frederik Vanderhaegen]

Hey,

I've written a http sniffer that monitors the traffic on port 80.
This application works like it should be but there is one small problem.
Suppose I run the application and I surf to www.google.be then I retrieve
all the HTTP messages (Response and Request)
but the content of the message is gzip encoded (http compression).

Here you see the original message:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Server: GWS/2.1
Content-Length: 1763
Date: Sun, 01 Oct 2006 09:22:42 GMT

< ÿWërÛ¸þï§@èF ×'(ÛqÖµDf'l.în ÓNsíL[·<$±
-{=~zþècì<õ? %1-´òX"Às¿|8~¦äÑ ¼ sFó%.1Õ-5»
½D
ÂOÍMiW¡gàÚ-qF',*
&üùÓÛñTÍ
3¢wRææ[͵¹±?OÆã½X¦7#"Z è¨MSÛEZ3Z2~sN£|¤©Ðc
Sew{f¿Âùñ´ºÆ åm"¹TçûÓir·7Gó À?ß>ëD±ÊàfU"Õ"1 L
¢³¡>ʤ.ÑöI6YN2 \è¡?»Û%|1¬I6 Jh:JÌ(IG:÷oY6\ó ²'æ ýÛ+ª¬~Håj"' )üüñâµ,+)'¼
;6Ï è"V0³üuèyÍfdY+ª hÙQýffeÄså[ò>"Þ!؍?,SÓ?Á þä»`äy¾¿Ùº<ò'÷ôø.ç7$ó·´$k¤ànóõùÛJøÜ Ø3+raÝú'ÐPûMÃOÞa'ópÍ}^ÜÆ1 ß.ZI©XãXøLæÙG úýïÿ¾þñmýþÃɏ ûàê|v§ÀÔJ£jÀÄ ÁÔîan»o®Dmý 8oj
ÜÏs±õbAØáL,Ü "IÈ.[zѳ~P·Ê²?PSY$ újëfY.TåL"'Ä= ÀòÂ"'Ñ<ÁäÂÒ¦1 K.,
§$ÎuE&òvUÑ4u« KMM§O'Ͳ¦Vw. Be.!WSVÑÜ7±Å Z¢9¶z,,ôlêó XV"¼i¤I"Ë?&?¬. ÑÁOÍ{a>'?BKÞ§ Z!<Ü<^RQOAÌ SbâIí-µK'ÖÏgÖÈhÎÊ-ÅzÁÑ6_ÐÄ¢Iß÷~\ºpíOÃ.[ÄÒY:å¶È\YE5ä X¢!§g)ßY9Êx a(ûkòZ¶äN¿÷? :Þôüz]^?čÄ"SÃb-ûX{X×Hß3
yuOûÉN7NZr¥tú ÛskÛçyfÿHfYái zW×Ö¢WÀsöÛoa_-NW±Âæ̤* m
44P.DÐ.F*÷Ay© ??°vk¨.я¿¹®È `ð0®"CfYØfi s?Ë{¤Ãâc
j°x7¬a,ÒáúOÙÐ[?zÿ[Ï¿mÅ.ø¶fQ<|Î$Kø 
~÷Ïï<dT-(ë¸?-FÄ7÷jv×aÙÑ
ûvy¶®ÝpGïÁ<MOMÒ@ĺsÝÿFdq é_¶`IÕ<#hP ¸bÂákÎ'EèµÖcSL Á´?CÉb@A«|³ ¢>r%ëJoÁS®ª¾þü Qý?0'¶f?,
úÿlÀ|n[ VºoxÔ,ê.~To¼Að? ÃÁYÙjׁ·¶)*T ˧è²-3?̏µØe8oÖ.â0 áøôikæ
t'Um^êÂ,¥x :¸f-8icÒÑ"ôsfÈÐé ³3W|§§Z|ÙRãYÑÌy "_%,@x
6lëÒu\2Óqõ?ÈØ^ w°\,H* oËÅ?çb'乶ÙSÉv G_M"¹?¦WT

When I copy the encoded content into notepad and save it as a gz file.
If I try to decompress it, I receive "CRC is invalid".
I've tried several things but nothing seems to work.

Has anybody how I can decompress the content of this message so I retrieve
the html code?

Thx in advance

Frederik

[Chris Fulstow]

Re: decompression problem

You can use the GZipStream class to decompress the response stream:
http://msdn2.microsoft.com/en-us/lib...zipstream.aspx

Or take a look at Fiddler:

http://www.fiddlertool.com/fiddler/

This is an HTTP sniffer with built-in support for gzip decompression.
Frederik Vanderhaegen wrote:

Hey,
>
I've written a http sniffer that monitors the traffic on port 80.
This application works like it should be but there is one small problem.
Suppose I run the application and I surf to www.google.be then I retrieve
all the HTTP messages (Response and Request)
but the content of the message is gzip encoded (http compression).
>
Here you see the original message:
>
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Server: GWS/2.1
Content-Length: 1763
Date: Sun, 01 Oct 2006 09:22:42 GMT
>
[SNIP]
>
When I copy the encoded content into notepad and save it as a gz file.
If I try to decompress it, I receive "CRC is invalid".
I've tried several things but nothing seems to work.
>
Has anybody how I can decompress the content of this message so I retrieve
the html code?
>
Thx in advance
>
Frederik

[Jon Slaughter]

Re: decompression problem


"Frederik Vanderhaegen" <frederik.vande rhaegen@pandora .bewrote in message
news:OIOfMwT5GH A.1012@TK2MSFTN GP05.phx.gbl...

Hey,
>
I've written a http sniffer that monitors the traffic on port 80.
This application works like it should be but there is one small problem.
Suppose I run the application and I surf to www.google.be then I retrieve
all the HTTP messages (Response and Request)
but the content of the message is gzip encoded (http compression).
>
Here you see the original message:
>
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Server: GWS/2.1
Content-Length: 1763
Date: Sun, 01 Oct 2006 09:22:42 GMT
>
< ÿWërÛ¸þï§@èF ×'(ÛqÖµDf'l.în ÓNsíL[·<$±
-{=~zþècì<õ? %1-´òX"Às¿|8~¦äÑ ¼ sFó %.1Õ-5»
½D
ÂOÍMiW¡gàÚ-qF',*
&üùÓÛñTÍ
3¢wRææ[͵¹±?OÆã½X¦7#"Z è¨MSÛ EZ3Z2~sN£|¤©Ðc
Sew{f¿Âùñ´ºÆ åm"¹TçûÓir·7Gó À?ß>ëD±ÊàfU"Õ"1 L
¢³¡>ʤ.ÑöI6YN2 \è¡?»Û%|1¬I6 Jh:JÌ(IG:÷oY6\ó ²'æ
ýÛ+ª¬~Håj"' )üüñâµ,+)'¼ ;6Ï
è"V0³üuèyÍfdY+ª hÙQýffeÄså[ò>"Þ!؍?,SÓ?Á þä»`äy¾¿Ùº< ò'÷ôø.ç7$ó·´
$k¤ànóõùÛJøÜ Ø3+raÝú'Ð Pû
MÃOÞa'ópÍ}^ÜÆ1 ß.ZI©XãXøLæÙG úýïÿ¾þñmýþÃɏ ûàê|v§ÀÔJ£jÀÄ ÁÔîan»o®Dmý 8oj
ÜÏs±õbAØáL,Ü "IÈ.[zѳ~P·Ê²?PSY$ újëfY.TåL"'Ä= ÀòÂ"'Ñ<ÁäÂÒ¦1 K.,
§$ÎuE&òvUÑ4u« KMM§O'Ͳ¦Vw. Be.!WSVÑÜ7±Å Z¢9¶z,,ôlêó
XV"¼i¤I"Ë?&?¬. ÑÁOÍ{a>'?BKÞ§ Z!<Ü<^RQOAÌ SbâIí-µK'ÖÏgÖÈhÎÊ-ÅzÁÑ6_ÐÄ¢
Iß÷~\ºpíOÃ.[ÄÒY:å¶È\YE5ä X¢!§g)ßY9Êx a(ûkòZ¶äN¿÷? :Þôüz]^?čÄ"SÃb-ûX{X×Hß3
yuOûÉN7NZr¥tú ÛskÛçyfÿHfYái zW×Ö¢WÀsöÛoa_-NW±Âæ̤* m 4
4P.DÐ.F*÷Ay© ??°vk¨.я¿¹®È `ð0®"CfYØfi s?Ë{¤Ãâc
j°x7¬a,ÒáúOÙÐ[?zÿ[Ï¿mÅ.ø¶fQ<|Î$Kø 
~÷Ïï<dT-(ë¸?-FÄ7÷jv×aÙÑ ûvy¶®ÝpGïÁ
<MOMÒ@ĺsÝÿFdq é_¶`IÕ<#hP ¸bÂákÎ'EèµÖcSL Á´?CÉb@A«|³ ¢>r%ëJoÁS®ª¾þü Qý?0'¶f?,
úÿlÀ|n[ VºoxÔ, ê.~To¼Að? ÃÁYÙjׁ·¶)*T ˧è²-3?̏µØe8oÖ.â0 áøôikæ
t'Um^êÂ,¥x :¸f-8icÒÑ"ôsfÈÐé ³3W|§§Z|ÙRãYÑÌy "_%,@x
6lëÒu\2Óqõ?ÈØ^ w°\,H* oËÅ?çb'乶ÙSÉv G_M"¹?¦WT
>
When I copy the encoded content into notepad and save it as a gz file.
If I try to decompress it, I receive "CRC is invalid".
I've tried several things but nothing seems to work.
>

Because when you cut and paste you are not copying all the characters. There
are many control characters that won't show up such as tabs(0x10 or
something), line feeds(0x9 I guess), etc...

If you stored the msg as a binary then you can use some hex editor to remove
the text header and it might work. (or just try to save the body as a binary
file with .zip extension and then open it and it should work)

[Joerg Jooss]

Re: decompression problem

Thus wrote Chris,

You can use the GZipStream class to decompress the response stream:
http://msdn2.microsoft.com/en-us/lib...ession.gzipstr
eam.aspx
>
Or take a look at Fiddler:

http://www.fiddlertool.com/fiddler/

This is an HTTP sniffer with built-in support for gzip decompression.

In .NET 2.0, HttpWebResponse can decompress HTTP messages automatically,
if you set HttpWebRequest. AutomaticDecomp ression to an appropriate value
(i.e. anything other than DecompressionMe thods.None).

Cheers,
--
Joerg Jooss
news-reply@joergjoos s.de