How to upload ugoira, the new pixiv animation system

@Type-kun: > Sacriven said: > > Uh, really? I can use my...

2016-09-21T03:28:29-04:00

Sacriven said:

Uh, really? I can use my bookmarklet just fine. My pixiv' appearance is still the same as always.

That means you didn't get switched yet. Looks like I was one of the few "lucky" ones, see topic #13205. Maaaaybe they will leave the old design as-is, but I doubt that somehow.

@Sacriven: > Type-kun said: > > Yeah, no preview is...

2016-09-21T03:10:02-04:00

Type-kun said:

Yeah, no preview is misleading, we should do something about that, but commentary is quoted properly when I try it. Did your pixiv design change recently, by chance? Bookmarklet can't work with whitecube yet.

Uh, really? I can use my bookmarklet just fine. My pixiv' appearance is still the same as always.

@Moonspeaker: > Type-kun said: > > Yeah, no preview is...

2016-09-21T00:44:14-04:00

Type-kun said:

Yeah, no preview is misleading, we should do something about that, but commentary is quoted properly when I try it. Did your pixiv design change recently, by chance? Bookmarklet can't work with whitecube yet.

Yes, the commentary showed up properly for me this time as well.

I hadn't noticed any recent Pixiv design change, but that's hardly conclusive.

@Type-kun: > Moonspeaker said: > > I'd initially tried...

2016-09-21T00:40:17-04:00

Moonspeaker said:

I'd initially tried using the bookmarklet, actually, but it didn't show any preview or quote the commentary, so I didn't know if I was doing it right by going straight from the HTML page rather than viewing the image directly (which my browser won't let me do from the Pixiv page).

So much for being clever.

Yeah, no preview is misleading, we should do something about that, but commentary is quoted properly when I try it. Did your pixiv design change recently, by chance? Bookmarklet can't work with whitecube yet.

@Moonspeaker: > Type-kun said: > > Well, Danbooru supports...

2016-09-20T23:05:10-04:00

Type-kun said:

Well, Danbooru supports ugoira upload without any reconversion for almost 2 years already. You just need to specify http://www.pixiv.net/member_illust.php?mode=medium&illust_id=57872141 in the source field, or use bookmarklet on that page.

I'd initially tried using the bookmarklet, actually, but it didn't show any preview or quote the commentary, so I didn't know if I was doing it right by going straight from the HTML page rather than viewing the image directly (which my browser won't let me do from the Pixiv page).

So much for being clever.

@Type-kun: > Sacriven said: > > Just want to know, is...

2016-09-20T11:28:31-04:00

Sacriven said:

Just want to know, is there any filesize limit when it comes to uploading ugoira? Last time I tried to upload a 36 MB ugoira but it failed...

Max file size is 35 MB for a file pulled from URL, 30MB for a file you upload. I think there was an issue on github to increase it, but I can't find it.

@Sacriven: Just want to know, is there any filesize limit...

2016-09-20T11:12:13-04:00

Just want to know, is there any filesize limit when it comes to uploading ugoira? Last time I tried to upload a 36 MB ugoira but it failed...

@Type-kun: > Moonspeaker said: > > I figured out today...

2016-09-20T10:51:15-04:00

Moonspeaker said:

I figured out today how to upload a reasonably small ugoira thanks to the link provided in Loc0's post one page back (if this post doesn't start page 11 of the thread). I think the information would be useful to have in the Upload help page, so one doesn't eventually have to dig too deep for it in the forum, but I also realize there's more to uploading ugoira posts than just that link and subsequent source link updating. Is it possible yet to condense the necessary info into a help page, or is it still being worked out?

Well, Danbooru supports ugoira upload without any reconversion for almost 2 years already. You just need to specify http://www.pixiv.net/member_illust.php?mode=medium&illust_id=57872141 in the source field, or use bookmarklet on that page.

@Moonspeaker: I figured out today how to upload a reasonably...

2016-09-20T10:02:32-04:00

I figured out today how to upload a reasonably small ugoira thanks to the link provided in Loc0's post one page back (if this post doesn't start page 11 of the thread). I think the information would be useful to have in the Upload help page, so one doesn't eventually have to dig too deep for it in the forum, but I also realize there's more to uploading ugoira posts than just that link and subsequent source link updating. Is it possible yet to condense the necessary info into a help page, or is it still being worked out?

@GenL: So there is BPG http://bellard.org/bpg/...

2015-02-13T19:21:46-05:00

So there is BPG
http://bellard.org/bpg/
http://bellard.org/bpg/animation.html

And Honeyview supports it now
http://www.bandisoft.com/honeyview/history/

Honeyview is Windows only, fairly popular already. They used libbpg.
libbpg is available on bpg page - libbpg-0.9.5.tar.gz
There are also binary tools for creating and viewing images, with documentation - bpg-0.9.5-win32.zip

Tested with image 48658471.
ugoira zip = 19.8 mb
gif = 41.8 mb
bpg = 513 kb
bpg lossless = 40.8 mb (seems glitched, strange color artifacts appear on the image)

Encoding time: no less than 30 minutes. Damn.

@Schrobby: I have an ugoira I can't upload due to file...

2014-11-21T09:13:11-05:00

I have an ugoira I can't upload due to file size, almost 40 megs. A higher upload limit sure would be nice, like 50 megs or something.

@Toks: Danbooru already does that. If you source...

2014-11-15T17:45:53-05:00

Danbooru already does that. If you source upload an ugoira it stores the original images and it also converts it to a video file as well (click "Save as video" underneath the ugoira).

@dolphinjet: Can Danbooru support video format? Those...

2014-11-15T16:08:21-05:00

Can Danbooru support video format?
Those ugoira can be shrinked to ~1/10 file size with h264 video codec.
GIF cannot do that and it loses quality more significantly.

Image to video conversion can be done by ffmpeg
Put the simple dos batch script into the image folder with the images, modify the settings, and then run it.

Here's an batch example,
Setting the image file name prefix, dimension, fps, crf (a video quality factor)
Note that 03d means 3-digit number, so %prefix%%%03d.jpg means prefix000.jpg and so on...

set prefix=img
set dim=1280x720
set fps=15
set crf=18

D:\MeGUI\tools\ffmpeg\ffmpeg.exe -f image2 -i "%prefix%%%03d.jpg" -r %fps% -s %dim% -vcodec libx264 -crf %crf% -preset slow %prefix%-output.mp4

^ As you can see, I've installed MeGUI and use it's ffmpeg included. You can install it separately.

In general, image to video conversion is simple, even simpler than making GIF.
The rest is server support. The script works for non-pixiv images too. People may convert any image sequences with their own PC.

@Schrobby: The op post has all sorts of things including...

2014-10-02T09:06:02-04:00

The op post has all sorts of things including scripts for getting those.

@ElectricSheep: Well, if you can make the .zip + timecodes...

2014-10-01T04:55:53-04:00

Well, if you can make the .zip + timecodes available in any reasonable format via http then whipping up a player should be fairly easy.

@Schrobby: Integrating the player and enable uploading the...

2014-09-30T09:13:48-04:00

Integrating the player and enable uploading the files in original form sure would be the best possible solution.

@ElectricSheep: > The really interesting bit is at the end: it...

2014-09-28T21:06:41-04:00

The really interesting bit is at the end: it turns out Pixiv actually released their player as open source! It's available at https://github.com/pixiv/zip_player. It looks like it would be pretty feasible to integrate this into Danbooru to give us native ugoira support.

It's not hard to implement anyway, using external libraries it only took me around 70LoC.

The real issue is that it it only works in the browser, you can't really save it to disk in animated form or post it anywhere else on the web.

The zip + frame times is good for archival purposes, but not convenient otherwise.

@Schrobby: Renamed the thread, the title now has ugoira in...

2014-09-28T11:33:27-04:00

Renamed the thread, the title now has ugoira in it.

@Dbx: > ElectricSheep said: > > Toks' and mine...

2014-09-28T06:01:49-04:00

ElectricSheep said:

Toks' and mine actually do set the actual VFR frame rate as a 2nd pass via mkvmerge.
The framerate in the ffmpeg encoding stage is not really important apart from encoding quality calculations, which can be tweaked based on any arbitrarily chosen framerate or a calculated average.

It might not have been clear, but I was indeed talking about frame rates for their effect on quality during encoding. One could at any point say his settings are "good enough," but I wondered if the choice of lowest crf meant the compromise on frame rate was unintentional.

@evazion: So I just came across these slidedecks from...

2014-09-27T19:18:19-04:00

So I just came across these slidedecks from tech talks the Pixiv developers gave on how Ugoira works:

https://speakerdeck.com/harukasan/ugoku-backend is partly in English and talks about the server-side intrastructure. There are some interesting stats about Pixiv in here, like the fact that Pixiv runs on 400 (!) servers.

http://tototoshi.github.io/slides/ugoira-tech-talks-ugoku-gazou-henkan/ is in Japanese. Not sure what it's says.

https://marcan.st/talks/2014_pixiv_ugoku_player/ briefly talks about how their HTML5 zip player works. The really interesting bit is at the end: it turns out Pixiv actually released their player as open source! It's available at https://github.com/pixiv/zip_player. It looks like it would be pretty feasible to integrate this into Danbooru to give us native ugoira support.