twitter uses HTTP/2.0 now, curl needs this flag to transparently decompress GZIPped content - tscrape - twitter scraper
HTML git clone git://git.codemadness.org/tscrape
DIR Log
DIR Files
DIR Refs
DIR README
DIR LICENSE
---
DIR commit edc04377d1528364b2cb7308c26bcf48bb0b7cdc
DIR parent d5e71f92f619c0f87bc6237caa7dd381be8e8f01
HTML Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 23 Apr 2016 12:37:17 +0200
twitter uses HTTP/2.0 now, curl needs this flag to transparently decompress GZIPped content
alternatively use HTTP/1.x and OpenBSD ftp -o - url | tscrape
Diffstat:
M README | 4 ++--
M tscrape.1 | 4 ++--
M tscrape.c | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
---
DIR diff --git a/README b/README
@@ -12,11 +12,11 @@ $ make
Usage
-----
-curl -s 'https://twitter.com/namehere/with_replies' | tscrape
+curl --compressed -s 'https://twitter.com/namehere/with_replies' | tscrape
or without retweets:
-curl -s 'https://twitter.com/namehere' | tscrape
+curl --compressed -s 'https://twitter.com/namehere' | tscrape
Why
DIR diff --git a/tscrape.1 b/tscrape.1
@@ -29,12 +29,12 @@ Twitter fullname (can be a retweet).
.El
.Sh EXAMPLES
.Bd -literal -offset left
-curl -s 'https://twitter.com/namehere/with_replies' | tscrape
+curl --compressed -s 'https://twitter.com/namehere/with_replies' | tscrape
.Ed
.Pp
or without retweets:
.Bd -literal -offset left
-curl -s 'https://twitter.com/namehere' | tscrape
+curl --compressed -s 'https://twitter.com/namehere' | tscrape
.Ed
.Sh SEE ALSO
.Xr curl 1
DIR diff --git a/tscrape.c b/tscrape.c
@@ -26,7 +26,7 @@ enum {
Username = 64
};
-/* for compatibility with libcs that don't have strlcat or strlcpy. The
+/* for compatibility with libc's that don't have strlcat or strlcpy. The
* functions are synced from OpenBSD */
#undef strlcat
size_t strlcat(char *, const char *, size_t);