Trip to Roztocze

So we went to Roztocze region, which is at the south-east of Poland. We stayed near Zwierzyniec--small town next to also small but much more famous Szczebrzeszyn. Szczebrzeszyn is peculiar because it contains `sz' (twice) `cz' and `rz' sounds, which are extremely difficult to pronounce for foreigners. So Szczebrzeszyn is near-impossible to pronounce for strangers. There is also a short poem by Jan Brzechwa (for children so every Poles knows it) which starts from ``W Szczebrzeszynie chrząszcz brzmi w trzcinie'' (cf short video)

We arrived to Szczebrzeszyn about 18:00 and made a quick 1 hour tour. Then we went to Zwierzyniec or to be precise to Sochy--a village 3km away from Zwierzyniec. Next day we walked around Sochy/Zwierzyniec and went by car to another small town Józefów. In Wednesday we visited famous Nature reserve ``Cascades on Tanew River'' (cf https://roztocze.love/p4871) and similar reserve on river Sopot nearby. In Thursday we completed a kayak trip from Zwierzyniec do Szczebrzeszyn on river Wieprz. The best kayak trip ever.

In Friday we moved west to Sandomierz, a medium-sized city at the Vistula river (160 km from Cracow); It is an old town in the center of an interesting region. We stay there for two days. We returned home in Sunday about 18:00

Photo-track of our is here and the photos are available at my flickr account.

The weather was sunny and warm Monday--Friday and cloudy but (almost) no rain in Saturday. Perfect.

Google Community Mobility Reports

Google has launched a new website that uses anonymous location data collected from users of Google products and services to show the level of social distancing taking place in various locations. The COVID-19 Community Mobility Reports web site will show population data trends of six categories: Retail and recreation, grocery and pharmacy, parks, transit stations, workplaces, and residential. The data will track changes over the course of several weeks, and as recent as 48-to-72 hours prior, and will initially cover 131 countries as well as individual counties within certain states. (cf. www.google.com/covid19/mobility/.)

The raports contains charts and comments in the form: NN% compared to baseline (in six above mentioned categories) where NN is a number. It is assumed the number is a percent change at the last date depicted (which accidentaly is a part of a filename). So for example a filename 2020-03-29_PL_Mobility_Report_en.pdf contains a sentence `Retail & recreation -78% compared to baseline` which (probably) means that (somehow) registered traffic at R&R facilities was 22% of the baseline. Anyway those six numbers was extracted for OECD countries (and some other countries) and converted to CSV file.

The conversion was as follows: first PDF files was downloaded with simple Perl script:

#!/usr/bin/perl
# https://www.google.com/covid19/mobility/
use LWP::UserAgent;
use POSIX 'strftime';

my $sleepTime = 11;

%OECD = ('Australia' => 'AU', 'New Zealand' => 'NZ',
'Austria' => 'AT', 'Norway' => 'NO', 'Belgium' => 'BE',
'Poland' => 'PL', 'Canada' => 'CA', 'Portugal' => 'PT',
'Chile' => 'CL', 'Slovak Republic' => 'SK',
## etc ...
);

@oecd = values %OECD;

my $ua = LWP::UserAgent->new(agent => 'Mozilla/5.0', cookie_jar =>{});
my $date = "2020-03-29";

foreach $c (sort @oecd) {
   $PP="https://www.gstatic.com/covid19/mobility/${date}_${c}_Mobility_Report_en.pdf";

   my $req = HTTP::Request->new(GET => $PP);
   my $res = $ua->request($req, "${date}_${c}_Mobility_Report_en.pdf");

   if ($res->is_success) { print $res->as_string; }
   else { print "Failed: ", $res->status_line, "\n"; }
}

Next PDF files was converted to .txt with pdftotext. The relevant fragments of .txt files looks like:

  Retail & recreation
+80%

-78%
compared to baseline

So it looks easy to extract the relevant numbers: scan line-by-line looking for a line with appropriate content (Retail & recreation for example). If found start searching for 'compared to baseline'. If found retrieve previous line:

#!/usr/bin/perl
$file = $ARGV[0];

while (<>) {   chomp();
  if (/Retail \& recreation/ ) { $rr = scan2base(); }
  if (/Grocery \& pharmacy/ ) { $gp = scan2base(); }
  if (/Parks/ ) { $parks = scan2base(); }
  if (/Transit stations/ ) { $ts = scan2base(); }
  if (/Workplaces/ ) { $wps = scan2base(); }
  if (/Residential/ ) { $res = scan2base();
     print "$file;$rr;$gp;$parks;$ts;$wps;$res\n";
     last;  }
}

sub scan2base {
  while (<>) {
   chomp();
   if (/compared to baseline/) {  return ($prevline); }
   $prevline = $_;
  }
}

Extracted data can be found here.

Alan Kurdi refugee boat named

My comment to the information which I have found recently that a (German) boat was named after Alan Kurdi: Refugee boat named in honour of Alan Kurdi more than three years after Syrian boy died at sea... (https://www.independent.co.uk/news/world/europe/alan-kurdi-refugee-rescue-boat-mediterranean-syria-sea-eye-a8773361.html)

Alan (Abdullah) Kurdi family tried to escape from Turkey but failed and his body was wash out in Turkey. There is a EU-Turkey `deal' on (stopping) migrants masterminded by Germany. The deal costs EU-taxpayer 6 billions EUR. If some Germans want more foreigners in their country they better will name a train (zug in German) from Stambul to Munich than a boat. They also should persuade their "mutter" to end the deal with president Erdogan. The train could for example be called Abdullah-Kurdi-Express (AKE).

The fact that Kurdi was a Turkish Kurd changes nothing. If Germany regards Turkish Kurds as war refugees-- as Turkish government in fact fights them--no problem to guarantee (all) of them appropriate status (before bording them to the above mentioned AKE Zug). BTW president Erdogan will be more than happy: he will get rid of Kurds (it is called ethic cleansing btw and is regarded as a sort of a war crime) with the hands of German plain fools (who regards themself of course as sensitive, tolerant etc...).

Geotagowanie zdjęć za pomocą ExifTool

Uwaga: Poniżej zakłada się, że program ExifTool jest używany w środowisku MS Windows.

Pobierz plik ExifTool.exe i umieść go w jakimś katalogu (SomeDir)

Otwórz wiersz polecenia (przejdź do wyszukiwania systemu Windows, wpisz polecenie cmd i otworzy się okno)

W oknie wiersza polecenia zmień katalog na SomeDir.

Geotagowanie zdjęć przy użyciu śladu w formacie GPX

Zdjęcia zrobione z nowoczesnych smartfonów są już geotagowane (jeśli opcja Lokalizacja jest włączona).

Dla aparatów nie wyposażonych w GPS, geotagowanie jest możliwe jeżeli dysponujemy śladem (zarejestrowanym za pomocą zewnętrznego odbiornika GPS.)

Synchronizowanie zdjęć ze śladem GPS

Pobierz plik GPX z odbiornika/rejestratora GPS (dokładna procedura zależy od typu odbiornika GPS) do katalogu SomeDir.

Zrób zdjęcie odbiornika GPS/ekranu lub wyświetlacza, na którym wyświetlany jest zegar z dokładnym czasem (na przykład ekranu komputera/smartfona).

Wpisz w wierszu poleceń:

exiftool -DateTimeOriginal IMG_20171021_113121.jpg 
Date/Time Original : 2017:10:29 11:52:46

Wartość taga DateTimeOriginal określa, że zdjęcie zostało wykonane o 11:52:46. Dokładny czas odczytujemy ze zdjęcia. Załóżmym że czas na zdjęciu wynosi 11:50:19 CET. Zatem DateTimeOriginal jest równe 11:52:46 CET a czas dokładny 11:50:19 CET, stąd różnica 11:32:46 - 11:50:19 = 2min 27s.

Wszystkie odbiorniki GPS wewnętrznie używają/rejestrują wyłącznie używając czasu UTC, nawet jeśli na ekranie pokazywany jest czas lokalny (Por. UTC_offset). Zwykle aparat wewnętrznie korzysta z czasu lokalnego. Jeśli tak jest, należy zadeklarować odpowiednie przesunięcie UTC_offset (w przeciwnym przypadku exiftool używa systemowego przesunięcia czasu względem UTC).

Skopiuj wszystkie zdjęcia do programu SomeDir.

Wpisz w wierszu poleceń:

exiftool -geosync=-02:27 '-geotime<${DateTimeOriginal}+01:00' -geotag my_gps.log SomeDir

Tag Geotime określa punkt w czasie dla którego obliczana jest pozycja GPS. Zapis -geotime<${DateTimeOriginal}+01:00 oznacza, że pozycja jest obliczana według czasu z taga DateTimeOriginal. Zaś dodanie +01:00 ustala CET jako strefę czasową (Przesunięcie +01:00 względem UTC)

Tag Geosync dodaje dodatkowy dryft czasu, jak wyjaśniono powyżej.

Utworzenie pliku KML

Opcja -p służy do generowania plików GPX i/lub KML.

exiftool -fileOrder gpsdatetime -p gpx.fmt SomeDir > SomeTripPhotos.gpx

albo

exiftool -fileOrder gpsdatetime -p kml.fmt SomeDir > SomeTripPhotos.kml

Pliki KML mogą być importowane do Map Google i Google Earth.

Uwaga: jeżeli mapa ma być wyświetlona nie lokalnie, ale na jakimś serwerze WWW (na przykład: http://www.gpsvisualizer.com/), zdjęcia muszą być przesłane na jakiś serwer, a następnie w pliku KML/GPX należy poprawić linki do tychże zdjęć. Dokładna procedura zależy od tego, który serwer jest używany do przechowywania zdjęć.

Przykładowe pliki formatów

gpx.fmt

#[HEAD]<?xml version="1.0" encoding="utf-8"?>
#[HEAD]<gpx version="1.0"
#[HEAD] creator="ExifTool $ExifToolVersion"
#[HEAD] xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#[HEAD] xmlns="http://www.topografix.com/GPX/1/0"
#[HEAD] xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
#[BODY]<wpt lat="$gpslatitude#" lon="$gpslongitude#">
#[BODY]  <name>$directory/$filename</name>
#[BODY]  <ele>$gpsaltitude#</ele>
#[BODY]  <desc>$directory/$filename</desc>
#[BODY]  <link href="$directory/$filename"><type>large</type></link>
#[BODY]  <link href="$directory/$filename"><type>thumbnail</type></link>
#[BODY]  <time>$gpsdatetime</time>
#[BODY]</wpt>
#[TAIL]</gpx>

kml.fmt

#[HEAD]<?xml version="1.0" encoding="UTF-8"?>
#[HEAD]<kml xmlns="http://earth.google.com/kml/2.0">
#[HEAD]  <Document>
#[HEAD]    <name>My Photos</name>
#[HEAD]    <open>1</open>
#[HEAD]    <Style id="Photo">
#[HEAD]      <IconStyle>
#[HEAD]        <Icon>
#[HEAD]          <href>http://maps.google.com/mapfiles/kml/pal4/icon38.png</href>
#[HEAD]          <scale>1.0</scale>
#[HEAD]        </Icon>
#[HEAD]      </IconStyle>
#[HEAD]    </Style>
#[HEAD]    <Folder>
#[HEAD]      <name>Waypoints</name>
#[HEAD]      <open>0</open>
#[BODY]      <Placemark>
#[BODY]        <description><![CDATA[<br/><table><tr><td>
#[BODY]        <img src='$directory/$filename'
#[BODY]          width='$imagewidth' height='$imageheight'>
#[BODY]        </td></tr></table>]]></description>
#[BODY]        <Snippet/>
#[BODY]        <name>$filename</name>
#[BODY]        <styleUrl>#Photo</styleUrl>
#[BODY]        <Point>
#[BODY]          <altitudeMode>clampedToGround</altitudeMode>
#[BODY]          <coordinates>$gpslongitude#,$gpslatitude#,0</coordinates>
#[BODY]        </Point>
#[BODY]      </Placemark>
#[TAIL]    </Folder>
#[TAIL]  </Document>
#[TAIL]</kml>

Powyższe łącznie z przykładami można pobrać klikając tutaj.

Geotagging images with ExifTool

Note: Below it is assumed that ExifTool is used in MS Windows environment.

Download the ExifTool.exe file and place it in some directory (SomeDir)

Open the Command line (go to windows search, type cmd and the window will open)

In the command line window, change directory to SomeDir.

Geotagging photos using GPX log

Photos taken with modern smartphones are already geotagged (if Location option is On.)

If camera is unable to add geocoordinates geotagging is still possible if GPS log is available (registered with some external GPS receiver/logger).

Synchronize photos with GPS log

Download the GPX file from your GPS receiver/logger (exact procedure depends on the type of GPS receiver) to SomeDir.

Take picture of GPS receiver/or any screen displaying accurate time (PC/Smartphone screen for example).

Type in command line:

exiftool -DateTimeOriginal IMG_20171021_113121.jpg 
Date/Time Original : 2017:10:29 11:52:46

So the file was created at 11:52:46 according to DateTimeOriginal tag. Exact time is depicted. and if for example the time depicted (accurate time) is 11:50:19 CET and DateTimeOriginal is 11:52:46 CET, then the time difference is 11:32:46 - 11:50:19 = 2min 27s, ie. accurate time is obtained by substracting 2min 27s from DateTimeOriginal.

All GPS receivers internally use/record UTC time only, even if they diplay local time (cf UTC_offset). Usually the camera internally uses local time. If it is the case one should declare appropriate UTC_offset (otherwise exiftool uses system offset).

Copy all photos to SomeDir.

Type in command line:

exiftool -geosync=-02:27 '-geotime<${DateTimeOriginal}+01:00' -geotag my_gps.log SomeDir

The Geotime tag specifies the point in time for which the GPS position is calculated. -geotime<${DateTimeOriginal}+01:00 means GPS position is calculated according to DateTimeOriginal and it is CET time (+01:00 offset from UTC)

The Geosync tag specifies extra time drift, as explained above.

Generate KML file

Exiftool -p option have to be used to output files in GPX and/or KML.

exiftool -fileOrder gpsdatetime -p gpx.fmt SomeDir > SomeTripPhotos.gpx

or

exiftool -fileOrder gpsdatetime -p kml.fmt SomeDir > SomeTripPhotos.kml

KML files can be imported to Google maps and or Google earth.

Note: If such a map is not displayed locally but upload to some WWW server (http://www.gpsvisualizer.com/ in the case of GPX file for example), photos have to be uploaded to some server first and then URL to photos fixed. Exact procedure depends on which server is used for storing photos.

Example format files

gpx.fmt

#[HEAD]<?xml version="1.0" encoding="utf-8"?>
#[HEAD]<gpx version="1.0"
#[HEAD] creator="ExifTool $ExifToolVersion"
#[HEAD] xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#[HEAD] xmlns="http://www.topografix.com/GPX/1/0"
#[HEAD] xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
#[BODY]<wpt lat="$gpslatitude#" lon="$gpslongitude#">
#[BODY]  <name>$directory/$filename</name>
#[BODY]  <ele>$gpsaltitude#</ele>
#[BODY]  <desc>$directory/$filename</desc>
#[BODY]  <link href="$directory/$filename"><type>large</type></link>
#[BODY]  <link href="$directory/$filename"><type>thumbnail</type></link>
#[BODY]  <time>$gpsdatetime</time>
#[BODY]</wpt>
#[TAIL]</gpx>

kml.fmt

#[HEAD]<?xml version="1.0" encoding="UTF-8"?>
#[HEAD]<kml xmlns="http://earth.google.com/kml/2.0">
#[HEAD]  <Document>
#[HEAD]    <name>My Photos</name>
#[HEAD]    <open>1</open>
#[HEAD]    <Style id="Photo">
#[HEAD]      <IconStyle>
#[HEAD]        <Icon>
#[HEAD]          <href>http://maps.google.com/mapfiles/kml/pal4/icon38.png</href>
#[HEAD]          <scale>1.0</scale>
#[HEAD]        </Icon>
#[HEAD]      </IconStyle>
#[HEAD]    </Style>
#[HEAD]    <Folder>
#[HEAD]      <name>Waypoints</name>
#[HEAD]      <open>0</open>
#[BODY]      <Placemark>
#[BODY]        <description><![CDATA[<br/><table><tr><td>
#[BODY]        <img src='$directory/$filename'
#[BODY]          width='$imagewidth' height='$imageheight'>
#[BODY]        </td></tr></table>]]></description>
#[BODY]        <Snippet/>
#[BODY]        <name>$filename</name>
#[BODY]        <styleUrl>#Photo</styleUrl>
#[BODY]        <Point>
#[BODY]          <altitudeMode>clampedToGround</altitudeMode>
#[BODY]          <coordinates>$gpslongitude#,$gpslatitude#,0</coordinates>
#[BODY]        </Point>
#[BODY]      </Placemark>
#[TAIL]    </Folder>
#[TAIL]  </Document>
#[TAIL]</kml>

The above with examples can be download from here

Chancellor Merkel victory for a visual person

Change in number of seats won by party (AfD is brown of course regardless official party colors :-):-)

library(ggplot2)

df <- read.csv("de.csv", sep = ';', header=T, na.string="NA");

ggplot(df, aes(x=party, y=diff, fill=party )) +
geom_bar(stat="identity") +
geom_text(aes(label=diff), vjust=-0.5) +
labs(x = "", y="change") +

ggtitle("German elections results (#MP change)") +

## AfD is brown regardless official party colors :-)
scale_fill_manual(values=c("#8B4513", "#56B4E9",
"yellow", "green", "red", "#ff6666") )

Visiting WW2 Museum in Gdańsk

Museum of the Second World War (Muzeum II Wojny Światowej) was opened in March 2017 midst open dispute between the Museum director (a former director now) and deputy PM/culture minister Gliński. Officially the dispute concerns the political message (the government insists on more patriotic education and more attention to suffering and heroism of Poles/Poland while the former director wanted a more universal message of horrendous sufferings of all involved nations.)

While generally I agree with the line of the former director I do not like/recommend the museum at all. It is very, very poor. And it is poor as it lacks interesting exhibits---which is more than visible from the very beginning (cf. photos). Lot of empty rooms, or huge showcases with almost nothing inside or boring stuff (everyday objects: spoons, plates, bicycles etc...). Many replicas/models, very often ridiculously small (ship 30cm long or a tank even smaller). Some stuff not particularly relevant (replica of pre-war Warsaw street or drawings from children competition "How you imagine a war")

In my opinion for a person with a decent knowledge of the subject visiting the museum would be a pure waste of time (and money---less important as the tickets are pretty cheap).

Installing Ubuntu in Asus X205TA

Ubuntu 16.04 @ X205TA

MS Windows broke down making the box unusable (no keybord/mouse/WiFi etc...) and as Google says it is not easy to recover the system on this particular machine I try Ubuntu, which was installed within 30 minutes (including 20 minutes for copying the files)

Sound is not working, but I suppose it is just a problem of installing proper drivers. After intalling the system 21Gb of 30Gb memory is free, while it was only 10Gb left on Windows (due to 10Gb non-removable recovery partition).

Still any doubts which system is better?

Weight of RWC players

Scrapping various Web pages I managed to gather data on players participating in last 4 Rugby World Cups. Is there a trend in body mass of rugby players participating in RWC tournaments?

Using Plotly API via Perl script (described here Box-plot chart with plot.ly and Perl API) I can quickly plot series of boxplots:

# cpan install WebService::Plotly in case  WebService::Plotly is not installed
plotly_boxplot.pl -col=5 -by=0 -title='RWC players by weight' -sep=';' rwc1999-2015.csv

Resulting boxplots and data can be viewed here.

There is a cold summer this year in Sopot

Temp in Sopot/July 2015

The following CSV (on-demand generated from raw data with simple Perl script) file contains outdoor temperature registred every hour starting from 2010 (with DS18B20 sensor):

dayhr;No;y2010;y2011;y2012;y2013;y2014;y2015;day30
d070100;001;14.6;17.5;14.9;10.1;12.9;12.2;0
d070101;002;13.4;16.7;14.1;10.1;12.8;12.5;3600
d070102;003;12.8;16.3;14.3;10.2;12.7;12.1;7200

dayhr is a label and day30 denotes number of seconds from the beginning od the period (for the first observation day30 equals 0, for the second 3600 etc.) The chart was produced with the following custom R script:

require(ggplot2)
library(scales)
number_ticks <- function(n) {function(limits) pretty(limits, n)}

d <- read.csv("july-by-day.csv", sep = ';',  header=T, na.string="NA");

datestart <- ISOdate(2015, 7, 1, tz = "");
d30 <- datestart + d$day30;
d[,"d30"]  <- d30;

ggplot(d, aes(x = d30)) +
  geom_line(aes(y = y2015, colour = 'y2015'), size=.3) +
  geom_line(aes(y = y2014, colour = 'y2014'), size=.3) +
  geom_smooth(aes(y = y2015, colour = 'y2015'), size=1) +
  geom_smooth(aes(y = y2014, colour = "y2014"), size=1) +
  ylab(label="Temp [C]") +
  xlab(label="Observation") +
  scale_y_continuous(breaks=number_ticks(15)) +
  scale_x_datetime(breaks = date_breaks("5 days")) +
  theme(legend.title=element_blank()) +
  ggtitle("Temperature in July in Sopot") +
  theme(legend.position=c(.6, .9)) +
  theme(legend.text=element_text(size=12));

ggsave(file="Temp-M7-2015.pdf", width=15, height=8)

Uploading pictures to Picasaweb with Perl/LWP::UserAgent

Previously described bash script allows for uploading a file to PicasaWeb only. With the following (simplified) Perl script one can upload pictures as well as upload with metadata (title description and tags) or create/list albums:

#!/usr/bin/perl

use strict;

use LWP::UserAgent;
use Getopt::Long;
use File::MimeInfo;
use XML::LibXML;

my $AlbumId ="6170354040646469009";
my $profileID="default";

my $Action = 'u'; ## x| u | c | l (default action is Upload)
my $entryTitle = ''; my $entryDescription = '';
my $entryKeywords = '';
my $ActionUpload =''; my $ActionList = '';
my $ActionCreate = ''; my $ActionXload = '';
my $ImageFile = '';
my $dummyReq='';

GetOptions("xload" => \$ActionXload, "upload" => \$ActionUpload,
  "list" => \$ActionList, "create" => \$ActionCreate,
  "title=s" => \$entryTitle, "description=s" => \$entryDescription,
  "keywords=s" => \$entryKeywords,
  "file=s" => \$ImageFile,
  "album=s" => \$AlbumId, ## UploadFile to Album
) ;

## Determine action:
if ( $ActionUpload ) {$Action = 'u'} elsif ( $ActionList ) { $Action = 'l'}
elsif ( $ActionCreate ) { $Action = 'c'}
elsif ( $ActionXload ) { $Action = 'x'}

OAuth 2.0 authorization is handled with Python script oauth2picasa.py. The script is an adapted/copy-pasted fragment of code borrowd from picasawebsync:

### Authenticate with external script (oauth2picasa.py):
my $ACCESS_TOKEN=`oauth2picasa.py`;
chomp($ACCESS_TOKEN);
print STDERR "*** AccessToken: $ACCESS_TOKEN [AlbumId: $AlbumId]\n";

my $req ; my $blog_entry ;

my $picasawebURL = "https://picasaweb.google.com/data/feed/api/user/$profileID";

if ( $Action eq 'c' ) {## Action: create album
  $req = HTTP::Request->new(POST => $picasawebURL );
  $req->header( 'Content-Type' => 'application/atom+xml' );

  $blog_entry = "<entry xmlns='http://www.w3.org/2005/Atom'
     xmlns:media='http://search.yahoo.com/mrss/'
     xmlns:gphoto='http://schemas.google.com/photos/2007'>"
  . "<title type='text'>$entryTitle</title>"
  . "<summary type='text'>$entryDescription</summary>"
  . "<media:group><media:keywords>$entryKeywords</media:keywords></media:group>"
  . "<category scheme='http://schemas.google.com/g/2005#kind'
     term='http://schemas.google.com/photos/2007#album'></category></entry>";

    $req->content($blog_entry);
} 
elsif ( $Action eq 'l' ) {## Action: list albums
  $req = HTTP::Request->new(GET => $picasawebURL );

} 
elsif ( $Action eq 'u' ) {## Action: Upload 1 photo w/o metadata
  my $mimeType = mimetype($ImageFile);

  ## https://developers.google.com/picasa-web/docs/2.0/developers_guide_protocol
  $req = HTTP::Request->new(POST => "$picasawebURL/albumid/$AlbumId" );
  $req->header( 'Content-Type' => "$mimeType" );
  $req->header( 'Slug' => "$ImageFile" );

  ## http://www.perlmonks.org/?node_id=131584
  open(FILE, $ImageFile);
  $req->content(join('',<FILE>));
  close(FILE);
}

To upload the binary image data along with its metadata, use MIME content type "multipart/related"; send photo metadata in one part of the POST body (Content-Type: application/atom+xml), and binary-encoded image data in another part. This is the preferred approach according to Picasa Web Albums Data API Picasa Web Albums Data API

elsif ( $Action eq 'x' ) {## Action: Upload 1 photo with metadata
  # https://groups.google.com/forum/#!topic/google-picasa-data-api/2qRfP0EIFhk
  my $mimeType = mimetype($ImageFile);

  $req = HTTP::Request->new(POST => "$picasawebURL/albumid/$AlbumId" );
  $req->header( 'Content-Type' => "multipart/related" );

  open(FILE, $ImageFile);
  my $add_photo_metadata = "<entry xmlns='http://www.w3.org/2005/Atom' 
     xmlns:media='http://search.yahoo.com/mrss/'>"
  . "<title type='text'>$entryTitle</title>"
  . "<summary type='text'>$entryDescription</summary>"
  . "<media:group><media:keywords>$entryKeywords</media:keywords></media:group>"
  . "<category scheme='http://schemas.google.com/g/2005#kind' 
     term='http://schemas.google.com/photos/2007#photo'></category></entry>";

  my $add_photo_data = join('',<FILE>); close(FILE);

  ## http://www.perlmonks.org/?node_id=131584
  $req->add_part(HTTP::Message->new(['Content-Type'
      => 'application/atom+xml'], $add_photo_metadata));
  $req->add_part(HTTP::Message->new(['Content-Type'
      => "$mimeType"], $add_photo_data));
}

$req->header( 'Authorization' => "Bearer $ACCESS_TOKEN" );
$req->header( 'GData-Version' => '2' );

## ### ###
my $res ;
my $ua = LWP::UserAgent->new;

$res = $ua->request($req);

if ($res->is_success) {
   my $decoded_response = $res->decoded_content;
   print "*** OK *** $decoded_response\n";
}

Usage:

Upload with metadata

picasaweb.pl -xload -title PICTURE-TITLE -descr DESCRIPTION \
  -keywords 'TAG1,TAG2' -file FILE.jpg -album ALBUMID

Upload w/o metadata:

picasaweb.pl -upload -file FILE.jpg -album 12345

Create album:

picasaweb.pl -create -title ALBUM-TITLE -descr DESCRIPTION \
   -keywords 'TAG1,TAG2'

List album:

picasaweb.pl -list

Source code: picasaweb.pl

Uploading pictures to Picasaweb with Curl

My (simplified) old bash script for uploading images to Picasaweb (cf. Using cURL to interact with Google Data services and PicasaUploader):

USERNAME=SomeUsername # GoogleAccountUsername
PASSWORD=SomePasswd # GoogleAccountPasswd

MY_PIC="$1" ## filename of the picture to upload ##
ALBUM_ID=6008849823888405298 # ID of picasaweb album
PIC_TITLE=`basename $MY_PIC` # filename w/o extension
PIC_TYPE=`file -b --mime-type "$MY_PIC"`

AUTH_KEY=$( curl -s https://www.google.com/accounts/ClientLogin -d Email="$USERNAME"@gmail.com \
  -d Passwd="$PASSWORD" -d accountType=GOOGLE \
  -d source=Google-cURL-Example -d service=lh2 | awk -F\= '/^Auth=/{print $2}' )
ALBUM_XML="http://picasaweb.google.com/data/feed/api/user/$USER_ID/albumid/$ALBUM_ID?authkey=$AUTH_KEY"

URL=`curl -s --request POST --data-binary "@$MY_PIC" --header "Slug: $PIC_TITLE" \
 --header "Content-Type: $PIC_TYPE" \
 --header "Authorization: GoogleLogin auth=$AUTH_KEY" "$ALBUM_XML" | \
sed 's/.*media:content url='"'"'\([^'"'"']*\).*media:thumbnail url='"'"'\([^'"'"']*\).*/\1/'`

URL contains url of uploaded picture (obtained with piping curl's output via sed).

This script stoped working recently as Google no longer supports OAuth 1.0 (cf. Google Identity Platform). Fortunately it was pretty easy to modify it to support OAuth 2.0:

ACCESS_TOKEN=$(oauth2picasa.py) ## Acces token is managed with Python's script now
## Note that ALBUM_XML URL starts now from https:// now
ALBUM_XML="https://picasaweb.google.com/data/feed/api/user/$USER_ID/albumid/$ALBUM_ID"

URL=`curl -s --request POST --data-binary "@$MY_PIC" \
--header "GData-Version: 2" --header "Slug: $PIC_TITLE" \
--header "Content-Type: $PIC_TYPE" -H "Authorization: Bearer $ACCESS_TOKEN" $ALBUM_XML |  \
sed 's/.*media:content url='"'"'\([^'"'"']*\).*media:thumbnail url='"'"'\([^'"'"']*\).*/\1/'`

NOTE: OAuth 2.0 authorization is handled with tiny Python script oauth2picasa.py. The script is an adapted/copy-pasted fragment of code borrowd from picasawebsync:

#!/usr/bin/python

import os
import time
import httplib2
## https://github.com/google/oauth2client
## installed with pip install --upgrade oauth2client (or some other way)
from oauth2client import client

def oauthLogin():
        # using http://stackoverflow.com/questions/20248555/list-of-spreadsheets-gdata-oauth2/29157967#29157967
        from oauth2client.file import Storage

        filename = os.path.join(os.path.expanduser('~'), ".picasawebsync")
        client_secrets = os.path.join(os.path.expanduser('~'), ".config", "picasawebsync.json")

 storage = Storage(filename)
        credentials = storage.get()
        if credentials is None or credentials.invalid:  
         flow = client.flow_from_clientsecrets(client_secrets,
         scope='https://picasaweb.google.com/data/',
         redirect_uri='urn:ietf:wg:oauth:2.0:oob')     
                auth_uri = flow.step1_get_authorize_url()       
                print 'Authorization URL: %s' % auth_uri
                auth_code = raw_input('Enter the auth code: ')
                credentials = flow.step2_exchange(auth_code)
                storage.put(credentials)
        if credentials.access_token_expired:
                credentials.refresh(httplib2.Http())
                
        return credentials.access_token

# start of the program

gd_client = oauthLogin()

print '%s' % gd_client

How to create an OAuth 2.0 client ID in the Google Developers Console is described here. Choose Client ID for native application. Next download JSON file, rename it to picasawebsync.json and move it to .config directory in your HOME directory (or modify the script).

First time used oauth2picasa.py prompts to sign on to google and paste the link back to authenticate.

Source code: picasa_upld.sh and oauth2picasa.py.

My old Canon LIDE 25 scanner works only once

My old Canon CanoScan LIDE 25 scanner is not working properly with my new hardware (Intel NUC) and Fedora 21.

I suspected Fedora 21 bug, but the problem is with the hardware, exactly as describe here. Works once and got stuck when scanning next.

It is claimed problem is solved in version 1.025 of Sane. To compile Sane one has to install libusb-devel package first:

yum install libusb-devel

Then:

git clone git://git.debian.org/sane/sane-backends.git
cd sane-backends

# I would like to install whole stuff in /usr/local
# without removing original fedora package
./configure --prefix=/usr/local --sysconfdir=/usr/local/etc \
    --localstatedir=/var BACKENDS=plustek
make
make install

Sane works now.

I would like to avoid mixing original Sane files with the new ones, so I use --prefix and --sysconfdir options with appropriate values. New Sane will be installed in /usr/local and it will be run by default--no need to remove original Fedora packages.

Sorry, Charlie Hebdo

It seems Charlie Hebdo massacre is a big surprise in France but what shock me even more is that the plain fact that France is at war with ISIS (cf French planes carry out air strikes on Isis targets in Iraq) was so efficiently obliterated from the awarness of the french populations, that the attack is met with naive incredulity and the reaction to it is so infantile.

The victims are presented now as defenders of freedom of speech which is not the case. They just produced stupid pictures, went to war with religion and (to their surprise) the religion retaliated. True free speach hero is (among many others) Norman Finkelstein who comments Charlie Hebdo massacre with the picture reprinted here.

New Year in Vienna

With ZK and his family we have spent 3 days in Vienna. If you are interested click the map below to see journey GPX track and photos.

We visited Schönbrunn Palace (a former imperial summer residence), Stephansdom, and Kunsthistorisches museum (where one can see Pieter Bruegel's famous paintings including: The Tower of Babel, The Fight Between Carnival and Lent, and The Hunters in the Snow).

We also visited Flak Towers which are enormous concrete bunkers constructed during 2nd world war. FLAK stands for Flug Abwehr Kanone (or Flugzeugabwehrkanone) and Flak Towers were a part (not particularly effective--mobile system is always better for obvious reason) of German air defence system intended to protect cities and residential areas from Allied air raids. Their construction was planned in September 1942 and Vienna was the third city after Berlin and Hamburg that got them.

The Flak Towers fulfilled two purposes (cf. Flak-Türme Towers, Vienna: Nazi concrete heritage at Vienna's heart): They held cannons and spotlights that should fight airplanes from the ground; and they were important bunkers with an autonomous electricity, air and water supply system. They were built in pairs, comprising of larger Gefechtsturm (Combat Tower) and a smaller Leitturm (Fire-control tower). The three pairs of Vienna can be found in the Augarten in the second district, the Arenberg Park in the third district and one each in the Esterhazypark in the sixth and the Stiftskaserne in the seventh.

We visited Augarten and Arenberg towers.

Trip to Georgia

I have just returned from a week-long trip to Georgia. We have visited Borjomi, Tbilisi, Kazbegi (Stepancminda) and Kutaisi (click the map to see journey track and photos).

On arriving to Kutaisi airport we went straight to the famous Borjomi resort where Russian Emperors once rested. We walked in the park tasting Borjomi mineral spring water, and swimming in hot water pool. From Borjomi we visited cave-city and monastery ensemble Vardzia where there was other hot water pool (with even hotter water). After 2 days in Borjomi we went to Tbilisi where we stayed 1 day (walking in old town). Next morning we drove to Caucasus mountains along Georgian military road via ancient Capital of Georgia -- Mtskheta (unfortunately we did not stop there). We stayed 2 days in Kazbegi where we hiked to Gergeti Trinity Church at 2170m and attended rugby union game Kazbegi vs Tbilisi Crusaiders (among other things:-). Finally we returned to Kutaisi where we visited Gelati and Motsameta Monastyrs as well as Sataplia caves...

We stayed in Leo's Homestay/Guesthouse in Borjomi, Why Not? Legend Hostel in Tbilisi and Hostel Kutaisi by Kote in Kutaisi. We recommend all of them.

The guesthouse we stayed in Kazbegi is not recommended so we do not mention it:-(

More details later...

Batch upload to Picasa

The problem: upload photos to Picasa (trivial); 2) scale them if neccessary before upload; 3) if photos contains some EXIF tags (geotags in particular) copy these tags to Picasa as well.

To achieve the above I use: googleCL (upload), convert (from ImageMagick bundle for scaling) exiftool (for metadata extraction/manipulation).

Using convert the script below (jpgresize.sh) scale picture to 2048 pixels along longest side:

#!/bin/bash
# Scale pictures to 2048 pixels along longest side for upload to Picasa
# Photos below 2048 x 2048 pixels do not count towards storage limit
# (cf. https://support.google.com/picasa/answer/6558?hl=en )
#
PICASA_FREE_LIMIT=2048

while test $# -gt 0; do
  case "$1" in
    -o)  shift; OUT_FILE="$1";;
   -o*)  OUT_FILE="`echo :$1 | sed 's/^:-o//'`";;
     *)  FILE="$1";;
  esac
  shift
done

if [ -z "$OUT_FILE" ] ; then
    my_pic="${FILE%.*}_s.${FILE#*.}"
else
    my_pic="$OUT_FILE";
fi 

if [ -f "$FILE" ] ; then

  echo "** converting $FILE to $my_pic ***"

  SIZE="2048x"

  my_pic_width=`exiftool -ImageWidth "$FILE" | awk '{print $NF}'`
  my_pic_height=`exiftool -ImageHeight "$FILE" | awk '{print $NF}'`

  if [[ ( -z "$my_pic_width" ) || ( -z "$my_pic_height" ) ]] ; then 
       echo "*** $FILE has 0 width and/or height ***"; exit ;
  fi

  ## http://www.imagemagick.org/Usage/resize/#resize
  if [[ ( "$my_pic_width" -gt "$PICASA_FREE_LIMIT" ) || \
      ( "$my_pic_height" -gt "$PICASA_FREE_LIMIT" ) ]] ; then
      if [ "$my_pic_width" -gt "$my_pic_height" ] ; then
        SIZE="${PICASA_FREE_LIMIT}x>"
	echo "*** $FILE width: $my_pic_width ; converting to $SIZE"
	convert "$FILE" -geometry $SIZE "$my_pic"
      else
        SIZE="x$PICASA_FREE_LIMIT>"
	echo "*** $FILE height: $my_pic_height ; converting to $SIZE"
	convert "$FILE" -geometry $SIZE "$my_pic"
      fi
   else
      ## File is too small copy the original:
      echo "*** $FILE has $my_pic_width in width; COPYING"
      cp "$FILE" "$my_pic"
   fi
else 
   echo "*** FILE $FILE not found! ***"
fi

Upload one picture to picasa with 1photo2picasa.sh

#!/bin/bash
#
# Upload photo to Picasa with googleCL
# It is assumed the photo contains UserComment GPSLatitude GPSLongitude GPSAltitude 
# Exif tags which are copied to Picasa (see below for more details)
# 
# Default album title:
ALBUMTITLE="???" 

echo "$0 -a AlbumTitle FILE-2-UPLOAD"

while test $# -gt 0; do
  case "$1" in
    -a)  shift; ALBUMTITLE="$1";;
   -a*)  ALBUMTITLE="`echo :$1 | sed 's/^:-a//'`";;
     *)  FILE="$1";;
  esac
  shift
done

AUTHOR=`exiftool -S -Artist $FILE`

if [ -z  "$AUTHOR" ] ; then 
   # It there is no Artist tag it is assumed photo was not tagged properly
   echo "*** ERROR: $FILE lacks Artist EXIF tag"
   exit;
else  
   ## Some tags are edited:
   TAGS=`exiftool -S -UserComment $FILE | awk '{ $1=""; for (i=1;i<=NF;i++) { if ($i ~ /http/) { $i=""}}; \
    gsub (/, +/, ",", $0); gsub (/ +,/, ",", $0);  gsub (/^ +| +$/, "", $0); print $0}'`
   GPSLat=`exiftool -S -c '%+.6f' -GPSLatitude $FILE | awk '{ print $2}'`
   GPSLon=`exiftool -S -c '%+.6f' -GPSLongitude $FILE | awk '{ print $2}'`
   GPSAlt=`exiftool -GPSAltitude $FILE -S -c "%.1f" | awk '{ if ($0 ~ /Below/) { print -$2} else {print $2}}'`

   PICASA_TAGS=""
   ## Concatenate all tags
   if [ -n  "$TAGS" ]   ; then  PICASA_TAGS="$TAGS";  fi
   if [ -n  "$GPSLat" ] ; then  PICASA_TAGS="$PICASA_TAGS,geo:lat=$GPSLat"; fi
   if [ -n  "$GPSLon" ] ; then  PICASA_TAGS="$PICASA_TAGS,geo:lon=$GPSLon"; fi
   if [ -n  "$GPSAlt" ] ; then  PICASA_TAGS="$PICASA_TAGS,geo:alt=$GPSAlt"; fi

   #  Upload to picasa:
   google picasa post --title "$ALBUMTITLE" --src="$FILE" --photo="$FILE" --tags="$PICASA_TAGS"
fi

Finally simple bash script upload2picassa.sh uses jpgresize.sh and 1photo2picasa.sh to upload all .jpg files from the current directory to picasa:

#!/bin/bash
# Upload all .jpg files (scaled tp 2048) to picasa 
#
echo "Uploading all .jpg files to album: $albumtitle"

albumtitle=$1

if [ -z "$albumtitle" ] ; then
  echo "Podaj ID albumu!";  exit 1
fi

for file in *.jpg; do

  echo "Uploading $file to $albumtitle album..."

  outfile="${file%.*}_s.${file#*.}"

  jpgresize.sh -p  -o $outfile $file  && 1photo2picasa.sh  -a $albumtitle $outfile
done

Letter from an Idiot

Not very long ago, there was a lot of LGBT propaganda before Sochi Olympics Games related to alleged Putin's war with LGBT Russians. Opinion leader Stephen Fry's open letter to PM Cameron/IOC was a good example of hysteria created then.

IMHO the letter was extremely stupid (as stupid as its creator), namely Putin was compared to Hitler and LGBT Russians to Jews in the 3rd Reich (He is making scapegoats of gay people, just as Hitler did Jews).

Unfortunately Putin started a real war, not a war with some scapegoats.... And of course comparing 6 millions murdered Jews with `suffering gays' is pure and simple grave robbery (taniec na trumnach in Polish).

Fry's letter is here

Great uptime of pinkaccordions.homelinux.org server

My tiny SheevaPlug ARM-based server has reached 366 days of uptime today. Proof included:

SheevaPlug is a sort of Raspberry Pi. One of the first such computers on the market, killed by Rpi a few years ago.

Box-plot chart with plot.ly and Perl API

The following Perl script reads data from a CSV file and draws a series of Box-Plots. Usage:

perl plotly_boxplot.pl -col=number -by=number -title=TITLE

where: -col=number -- column number containig variable to plot, -by=number -- column number containig grouping variable.

#!/usr/bin/perl
use WebService::Plotly;
use Getopt::Long;

# login to plotly script
require "$ENV{'HOME'}/bin/login2plotly.pl";

my $plotly = WebService::Plotly->new( un => $plotly_user, key => $plotly_key );

my $sep_sequence = ';';
my $col_number = -1;
my $by_col_number = -1;
my $chart_title='??Chart title??';
my $header='Y';
#my $boxpoints='outliers'; ## or all' | 'outliers' | False
my $USAGE="*** USAGE: -col=i -by=i -title=s -header=s -sep=s FILE *** \n";


# plot values from column 'col' grouped by column 'by'. If header is Y skip first row in data.
# Add title 'title'. Columns in csv data are separated by 'sep' (default ';')
GetOptions("col=i" => \$col_number, "by=i" => \$by_col_number, "title=s" => \$chart_title,
        'header=s' => \$header, 'sep=s' => \$sep_sequence, );
        ##'boxpoints=s' => \$boxpoints ) ;  ## this option not work!

if (($col_number == -1 ) || ($by_col_number == -1) ) { print $USAGE } 

while (<>) { chomp ($_); $nr++;
    if (($nr < 2) << ( $header eq 'Y' ) ) { next }
    $_ =~ s/"//g;
    my @fields = split(/$sep_sequence/, $_);
    push @{$data{$fields[$by_col_number]}}, $fields[$col_number];
    # http://stackoverflow.com/questions/3779213/how-do-i-push-a-value-onto-a-perl-hash-of-arrays
}

my @variants = sort keys %data;

print STDERR "*** No of rows scanned: $nr ***\n";
print STDERR "*** Groups found: @variants ($boxpoints) \n";
for $k (keys %data ) {
 print "$k"; push (@boxes, { y =>$data{$k}, type => 'box', #'boxpoints' => 'none',
  name => "$k" } ) }

my $layout = { 'title' => $chart_title };

my $response = $plotly->plot(\@boxes, layout => $layout );

my $url = $response->{url};
my $filename = $response->{filename};

print STDERR "*** done: filename: '$filename' url: '$url' ***\n"

Example: Age of Nobel Prize winners by discipline (grouping wariable) plot.ly/~tomasz.przechlewski/28/