heuzef
/
pannellum
mirror of https://github.com/mpetroff/pannellum.git
1
0
Fork 0
Code Issues 0 Releases 22 Wiki Activity
Browse Source

Add support for multires SHT and thumbnail previews. 

Spherical harmonic transform (SHT) previews are display instead of a solid
background color before the base multires tiles are loaded. If a thumbnail is
provided, it will be displayed instead of the SHT preview. If the thumbnail is
provided as a Base64-encoded string, the SHT preview may be briefly displayed,
since image loading is asynchronous, while the SHT preview is loaded
synchronously.
pull/1015/head
Matthew Petroff 3 years ago
parent
383e73cdbd
commit
9788fc6512
7 changed files with 497 additions and 11 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +17
    -0
      doc/json-config-parameters.md
  2. +65
    -0
      doc/sht-hash.md
  3. +1
    -1
      readme.md
  4. +326
    -4
      src/js/libpannellum.js
  5. +2
    -1
      utils/multires/Dockerfile
  6. +80
    -2
      utils/multires/generate.py
  7. +6
    -3
      utils/multires/readme.md

+ 17
- 0
doc/json-config-parameters.md View File

@@ -492,6 +492,23 @@ This specifies the size in pixels of the full resolution cube faces the image
tiles were created from.


#### `shtHash` (string)

Specifies the spherical-harmonic-transform-based preview hash. This is rendered
instead of the background color before the base set of cube faces are loaded.


#### `equirectangularThumbnail` (string or HTMLImageElement or ImageData or ImageBitmap)

Specifies a equirectangular preview thumbnail to be rendered instead of the
background color or SHT hash before the base set of cube faces are loaded. This
image can either be specified as a Base64-encoded string or as an object that
can be directly uploaded to a WebGL texture, e.g., `ImageData`, `ImageBitmap`,
`HTMLImageElement`, `HTMLCanvasElement` objects. If a Base64-encoded string is
used, the image size should be kept small, since it needs to be loaded with the
configuration parameters.



## Dynamic content specific options



+ 65
- 0
doc/sht-hash.md View File

@@ -0,0 +1,65 @@
# Spherical harmonic transform hash

This document specifies a spherical harmonic transform (SHT) hash, which is
intended to be a compact method of encoding a spherical panorama preview. It is
based on the [BlurHash specification](https://github.com/woltapp/blurhash/blob/master/Algorithm.md)
for DCT-based 2D image previews.

There are three steps for creating a SHT hash:

1. Calculate real spherical harmonic transform coefficients
2. Encode with compact binary encoding
3. Encode binary data as Base-83-encoded string

Spherical harmonics form an orthogonal basis for representing a function on the
sphere. Combined with coefficients for each harmonic, they can be used to
represent a frequency-space approximation of such a function, without boundary
effects. There are multiple normalization conventions for spherical harmonics;
the $4\pi$ convention is used here. Since JavaScript does not natively support
complex numbers, real harmonics with separate real sine and cosine coefficients
are used.

Spherical harmonics, $Y_{\ell m}$, are defined for $\ell \in \mathbb{Z}^+$,
with $|m| \leq \ell$. For each $Y_{\ell m}$, there is a corresponding
$f_{\ell m}$ coefficient. When these coefficients are represented in a pair of
matrices (one for the sine coefficient and one for the cosine coefficients)
with rows indexed by $\ell$ and columns indexed by $m$, the upper triangle of
the matrix is zero. Additionally, the $\ell = m = 1$ coefficient in the sine
matrix is always 1 in the $4\pi$ normalization convention, which is why it is
used here. Spherical harmonic coefficients are calculated separately for each
color channel. Once calculated, the sine and cosine coefficients are stored in
1D arrays using row-first ordering with the upper triangle of the matrices
excluded. The coefficient arrays also excludes the first row and column of
the coefficient matrices since their contents are always zero, except for the
$\ell = m = 1$ sine coefficient, which is always 1 (as previously mentioned).
The 1D cosine coefficient array is appended to the 1D sine coefficient array,
for each of the color channels.

The maximum coefficient magnitude is then found across the color channels, and
this value is used to normalize the coefficients in the range $[-1, 1]$. The
normalized coefficients are then multiplied by 9 and converted to integers,
thereby quantizing the coefficients as integer values. These signed integers in
the range $[-9, 9]$ are then converted to unsigned integers in the range
$[0, 18]$ by adding 9. For each coefficient, the color channel values are
packed into a single number in the range $[0, 6859]$ using
$R \cdot 19^2 + G \cdot 19 + B$. This number is then Base-83-encoded into a
pair of characters. Color is encoded and decoded using gamma-compressed sRGB
values, for simplicity.

The final SHT hash string is constructed by combining the Base-83-encoded
coefficients with a prefix. The first character in the prefix contains the max
$\ell$ value for the coefficients, encoded as Base 83. For Pannellum, this is
currently fixed at $\ell = 5$. The next character contains the maximum
coefficient value, which was used in the normalization. The value is divided by
255 to normalize it to the range $[0, 1]$. This value is then multiplied by 82
and quantized as an integer, before being Base-83 encoded. For $\ell = 5$ this
string is 74 characters in length.


## Base 83

A custom Base-83 encoding is used. Values are encoded individually, using one
or two digits, and concatenated together. Multiple-digit values are encoded in
big-endian order, with the most-significant digit first.

The character set used is `0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~`.

+ 1
- 1
readme.md View File

@@ -35,7 +35,7 @@ Configuration parameters are documented in the `doc/json-config-parameters.md` f
For final deployment, it is recommended that one use a minified copy of Pannellum instead of using the source files in `src` directly. The easiest method is to download the most recent [release](https://github.com/mpetroff/pannellum/releases) and use the pre-built copy of either `pannellum.htm` or `pannellum.js` & `pannellum.css`. If you wish to make changes to Pannellum or use the latest development copy of the code, follow the instructions in the _Building_ section below to create `build/pannellum.htm`, `build/pannellum.js`, and `build/pannellum.css`.

### Using `generate.py` to create multires panoramas
To be able to create multiresolution panoramas, you need to have the `nona` program installed, which is available as part of [Hugin](http://hugin.sourceforge.net/), as well as Python with the [Pillow](https://pillow.readthedocs.org/) package. Then, run
To be able to create multiresolution panoramas, you need to have the `nona` program installed, which is available as part of [Hugin](http://hugin.sourceforge.net/), as well as Python 3 with the [Pillow](https://pillow.readthedocs.org/) and [NumPy](https://numpy.org/) packages. The [pyshtools](https://shtools.github.io/SHTOOLS/) Python package is also recommended. Then, run

```
python3 generate.py pano_image.jpg


+ 326
- 4
src/js/libpannellum.js View File

@@ -36,6 +36,7 @@ function Renderer(container) {
container.appendChild(canvas);

var program, gl, vs, fs;
var previewProgram, previewVs, previewFs;
var fallbackImgSize;
var world;
var vtmps;
@@ -99,6 +100,18 @@ function Renderer(container) {
gl.deleteProgram(program);
program = undefined;
}
if (previewProgram) {
if (previewVs) {
gl.detachShader(previewProgram, previewVs);
gl.deleteShader(previewVs);
}
if (previewFs) {
gl.detachShader(previewProgram, previewFs);
gl.deleteShader(previewFs);
}
gl.deleteProgram(previewProgram);
previewProgram = undefined;
}
pose = undefined;

var s;
@@ -470,7 +483,10 @@ function Renderer(container) {
}

// Set parameters for rendering any size
gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
if (imageType != "cubemap" && image.width <= maxWidth && haov == 2 * Math.PI)
gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.REPEAT);
else
gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(glBindType, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(glBindType, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(glBindType, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
@@ -512,6 +528,124 @@ function Renderer(container) {
program.nodeCache = [];
program.nodeCacheTimestamp = 0;
program.textureLoads = [];

if (image.shtHash || image.equirectangularThumbnail) {
// Create vertex shader
previewVs = gl.createShader(gl.VERTEX_SHADER);
gl.shaderSource(previewVs, v);
gl.compileShader(previewVs);

// Create fragment shader
previewFs = gl.createShader(gl.FRAGMENT_SHADER);
gl.shaderSource(previewFs, fragEquirectangular);
gl.compileShader(previewFs);

// Link WebGL program
previewProgram = gl.createProgram();
gl.attachShader(previewProgram, previewVs);
gl.attachShader(previewProgram, previewFs);
gl.linkProgram(previewProgram);

// Log errors
if (!gl.getShaderParameter(previewVs, gl.COMPILE_STATUS))
console.log(gl.getShaderInfoLog(previewVs));
if (!gl.getShaderParameter(previewFs, gl.COMPILE_STATUS))
console.log(gl.getShaderInfoLog(previewFs));
if (!gl.getProgramParameter(previewProgram, gl.LINK_STATUS))
console.log(gl.getProgramInfoLog(previewProgram));

// Use WebGL program
gl.useProgram(previewProgram);

// Look up texture coordinates location
previewProgram.texCoordLocation = gl.getAttribLocation(previewProgram, 'a_texCoord');
gl.enableVertexAttribArray(previewProgram.texCoordLocation);

// Provide texture coordinates for rectangle
if (!texCoordBuffer)
texCoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1,1,1,1,1,-1,-1,1,1,-1,-1,-1]), gl.STATIC_DRAW);
gl.vertexAttribPointer(previewProgram.texCoordLocation, 2, gl.FLOAT, false, 0, 0);

// Pass aspect ratio
previewProgram.aspectRatio = gl.getUniformLocation(previewProgram, 'u_aspectRatio');
gl.uniform1f(previewProgram.aspectRatio, gl.drawingBufferWidth / gl.drawingBufferHeight);

// Locate psi, theta, focal length, horizontal extent, vertical extent, and vertical offset
previewProgram.psi = gl.getUniformLocation(previewProgram, 'u_psi');
previewProgram.theta = gl.getUniformLocation(previewProgram, 'u_theta');
previewProgram.f = gl.getUniformLocation(previewProgram, 'u_f');
previewProgram.h = gl.getUniformLocation(previewProgram, 'u_h');
previewProgram.v = gl.getUniformLocation(previewProgram, 'u_v');
previewProgram.vo = gl.getUniformLocation(previewProgram, 'u_vo');
previewProgram.rot = gl.getUniformLocation(previewProgram, 'u_rot');

// Pass horizontal extent
gl.uniform1f(previewProgram.h, 1.0);

// Create texture
previewProgram.texture = gl.createTexture();
gl.bindTexture(glBindType, previewProgram.texture);

// Upload preview image to the texture
var previewImage, vext, voff;
var uploadPreview = function() {
gl.useProgram(previewProgram);

gl.uniform1i(gl.getUniformLocation(previewProgram, 'u_splitImage'), 0);
gl.texImage2D(glBindType, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, previewImage);

// Set parameters for rendering any size
gl.texParameteri(glBindType, gl.TEXTURE_WRAP_S, gl.REPEAT);
gl.texParameteri(glBindType, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(glBindType, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(glBindType, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

// Pass vertical extent and vertical offset
gl.uniform1f(previewProgram.v, vext);
gl.uniform1f(previewProgram.vo, voff);

gl.useProgram(program);
};
if (image.shtHash) {
previewImage = shtDecodeImage(image.shtHash);
// Vertical extent & offset are chosen to set the top and bottom
// pixels in the preview image to be exactly at the zenith and
// nadir, respectively, which matches the pre-calculated Ylm
vext = (2 + 1 / 31) / 2;
voff = 1 - (2 + 1 / 31) / 2;
uploadPreview();
}
if (image.equirectangularThumbnail) {
if (typeof image.equirectangularThumbnail === 'string') {
if (image.equirectangularThumbnail.slice(0, 5) == 'data:') {
// Data URI
previewImage = new Image();
previewImage.onload = function() {
vext = 1;
voff = 0;
uploadPreview();
};
previewImage.src = image.equirectangularThumbnail;
} else {
console.log('Error: thumbnail string is not a data URI!');
throw {type: 'config error'};
}
} else {
// ImageData / ImageBitmap / HTMLImageElement / HTMLCanvasElement
previewImage = image.equirectangularThumbnail;
vext = 1;
voff = 0;
uploadPreview();
}
}

// Reactivate main program
gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertBuf);
gl.vertexAttribPointer(program.vertPosLocation, 3, gl.FLOAT, false, 0, 0);
gl.useProgram(program);
}
}

// Check if there was an error
@@ -562,6 +696,10 @@ function Renderer(container) {
gl.viewport(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight);
if (imageType != 'multires') {
gl.uniform1f(program.aspectRatio, canvas.clientWidth / canvas.clientHeight);
} else if (image.shtHash) {
gl.useProgram(previewProgram);
gl.uniform1f(previewProgram.aspectRatio, canvas.clientWidth / canvas.clientHeight);
gl.useProgram(program);
}
}
};
@@ -688,6 +826,41 @@ function Renderer(container) {
gl.drawArrays(gl.TRIANGLES, 0, 6);
} else {
// Draw SHT hash preview, if needed
var drawPreview = typeof image.shtHash !== 'undefined'
if (drawPreview && program.currentNodes.length >= 6) {
drawPreview = false;
for (var i = 0; i < 6; i++) {
if (!program.currentNodes[i].textureLoaded) {
drawPreview = true;
break;
}
}
}
if (drawPreview) {
gl.useProgram(previewProgram);
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.vertexAttribPointer(previewProgram.texCoordLocation, 2, gl.FLOAT, false, 0, 0);
gl.bindTexture(gl.TEXTURE_2D, previewProgram.texture);

// Calculate focal length from vertical field of view
var vfov = 2 * Math.atan(Math.tan(hfov * 0.5) / (gl.drawingBufferWidth / gl.drawingBufferHeight));
focal = 1 / Math.tan(vfov * 0.5);

// Pass psi, theta, roll, and focal length
gl.uniform1f(previewProgram.psi, yaw);
gl.uniform1f(previewProgram.theta, pitch);
gl.uniform1f(previewProgram.rot, roll);
gl.uniform1f(previewProgram.f, focal);

// Draw using current buffer
gl.drawArrays(gl.TRIANGLES, 0, 6);

gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertBuf);
gl.vertexAttribPointer(program.vertPosLocation, 3, gl.FLOAT, false, 0, 0);
gl.useProgram(program);
}

// Create perspective matrix
var perspMatrix = makePersp(hfov, gl.drawingBufferWidth / gl.drawingBufferHeight, 0.1, 100.0);
@@ -756,7 +929,7 @@ function Renderer(container) {
program.textureLoads.shift()();

// Draw tiles
multiresDraw();
multiresDraw(!image.shtHash);
}
if (params.returnImage !== undefined) {
@@ -837,13 +1010,15 @@ function Renderer(container) {
/**
* Draws multires nodes.
* @param {bool} clear - Whether or not to clear canvas.
* @private
*/
function multiresDraw() {
function multiresDraw(clear) {
if (!program.drawInProgress) {
program.drawInProgress = true;
// Clear canvas
gl.clear(gl.COLOR_BUFFER_BIT);
if (clear)
gl.clear(gl.COLOR_BUFFER_BIT);
// Determine tiles that need to be drawn
var node_paths = {};
for (var i = 0; i < program.currentNodes.length; i++)
@@ -1408,6 +1583,153 @@ function Renderer(container) {
canvas.width = Math.round(canvas.width / 2);
canvas.height = Math.round(canvas.height / 2);
}

// Data for rendering SHT hashes
var shtB83chars = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~',
shtYlmStr = 'Bf[ff4fff|ffff0fffffBo@Ri5xag{Jmdf2+WiefCs@Ll7+Vi]Btag6' +
'[NmdgCv=Ho9;Qk;7zWiF_GsahDy:ErE?Mn$5+SkS_AyWiD#-CuJ[Iqp6;Nnx?7*SlE$' +
'*BxR@FtPA?Jq+%7:NnF*zAzn?CwIG@Ft-Y9?IrG+vA%w:AzGR?Cx*IF@EuI,nA+$*9%' +
'Gu:A#xCR?ByJ-VB-*wA+J**9*ZBv:9%L.QD.*aB.O.v9-MF+$8,O:MG:*OD;a:UB:IO' +
':n9:Q:KJ;#IG=u-KE=Hs:MC?T:IO=wEL?#%FJ@K**FI@Y;HV=pDU?*sCS@S.uCR[m;H' +
'p=VDq?*SCs@s.QCt[r:Iw=OEz?#IF$@#*HF%@u:K$;KI+=uEK-=*sCM:?w:M+:HO.;a' +
'CU;:%OCn?:z.Q..Ha;.ODv?-yFG$@,$-V;-Hw=+JH*?*lBP:?%%,n=+J*?%GQ:=#NCt' +
'?;y++v=%O:=zGt?:xHI,@-u,*z=zX?:wI+@,tEY??%r-$*;xt@,tP=?$qG%[:xn.#-:' +
'u$[%qp];xnN?[*sl.y:-r-?yn$^+sks_=yoi:v=*o?;uk;[zoi,_+skh:s@zl[+pi];' +
'tkg][xmhg;o@ti^xkg{$mhf|+oigf;f[ff_fff|ffff~fffff',
shtMaxYlm = 3.317,
shtYlm = [];

/**
* Decodes an integer-encoded float.
* @private
* @param {number} i - Integer-encoded float.
* @param {number} maxVal - Maximum value of decoded float.
* @returns {number} Decoded float.
*/
function shtDecodeFloat(i, maxVal) {
return Math.pow(((Math.abs(i) - maxVal) / maxVal), 2) * (i - maxVal > 0 ? 1 : -1);
}

/**
* Decodes encoded spherical harmonic transform coefficients.
* @private
* @param {number} val - Encoded coefficient.
* @param {number} maxVal - Maximum value of coefficients.
* @returns {number[]} Decoded coefficients; one per color channel [r, g, b].
*/
function shtDecodeCoeff(val, maxVal) {
var quantR = Math.floor(val / (19 * 19)),
quantG = Math.floor(val / 19) % 19,
quantB = val % 19;
var r = shtDecodeFloat(quantR, 9) * maxVal,
g = shtDecodeFloat(quantG, 9) * maxVal,
b = shtDecodeFloat(quantB, 9) * maxVal;
return [r, g, b];
}

/**
* Decodes base83-encoded string to integers.
* @private
* @param {string} b83str - Encoded string.
* @param {number} length - Number of characters per integer.
* @returns {number[]} Decoded integers.
*/
function shtB83decode(b83str, length) {
var cnt = Math.floor(b83str.length / length),
vals = [];
for (var i = 0; i < cnt; i++) {
var val = 0;
for (var j = 0; j < length; j++) {
val = val * 83 + shtB83chars.indexOf(b83str[i * length + j]);
}
vals.push(val);
}
return vals;
}

/**
* Renders pixel from spherical harmonic transform coefficients.
* @private
* @param {number[]} flm - Real spherical harmonic transform coefficients.
* @param {number[]} Ylm - 4pi-normalized spherical harmonics evaluated for ell, m, and lat.
* @param {number} lon - Longitude (radians).
* @returns {number} Pixel value.
*/
function shtFlm2pixel(flm, Ylm, lon) {
var lmax = Math.floor(Math.sqrt(flm.length)) - 1

// Precalculate sine and cosine coefficients
var cosm = Array(lmax + 1),
sinm = Array(lmax + 1);
sinm[0] = 0;
cosm[0] = 1;
sinm[1] = Math.sin(lon);
cosm[1] = Math.cos(lon);
for (var m = 2; m <= lmax; m++) {
sinm[m] = 2 * sinm[m - 1] * cosm[1] - sinm[m - 2];
cosm[m] = 2 * cosm[m - 1] * cosm[1] - cosm[m - 2];
}

// Calculate value at pixel
var expand = 0,
cosidx = 0;
for (var i = 1; i <= lmax + 1; i++)
cosidx += i;
for (var l = lmax; l >= 0; l--) {
var idx = Math.floor((l + 1) * l / 2);
// First coefficient is 1 when using 4pi normalization
expand += idx != 0 ? flm[idx] * Ylm[idx - 1] : flm[idx];
for (var m = 1; m <= l; m++)
expand += (flm[++idx] * cosm[m] + flm[idx + cosidx - l - 1] * sinm[m]) * Ylm[idx - 1];
}

return Math.round(expand);
}

/**
* Renders image from spherical harmonic transform (SHT) hash.
* @private
* @param {string} shtHash - SHT hash.
* @returns {ImageData} Rendered image.
*/
function shtDecodeImage(shtHash) {
if (shtYlm.length < 1) {
// Decode Ylm if they're not already decoded
var ylmLen = shtYlmStr.length / 32;
for (var i = 0; i < 32; i++) {
shtYlm.push([]);
for (var j = 0; j < ylmLen; j++)
shtYlm[i].push(shtDecodeFloat(shtB83decode(shtYlmStr[i * ylmLen + j], 1), 41) * shtMaxYlm);
}
}

// Decode SHT hash
var lmax = shtB83decode(shtHash[0], 1)[0],
maxVal = (shtDecodeFloat(shtB83decode(shtHash[1], 1), 41) + 1) * 255 / 2,
vals = shtB83decode(shtHash.slice(2), 2),
rVals = [],
gVals = [],
bVals = [];
for (var i = 0; i < vals.length; i++) {
var v = shtDecodeCoeff(vals[i], maxVal);
rVals.push(v[0]);
gVals.push(v[1]);
bVals.push(v[2]);
}

// Render image
var lonStep = 0.03125 * Math.PI;
var img = [];
for (var i = 31; i >= 0; i--) {
for (var j = 0; j < 64; j++) {
img.push(shtFlm2pixel(rVals, shtYlm[i], (j + 0.5) * lonStep));
img.push(shtFlm2pixel(gVals, shtYlm[i], (j + 0.5) * lonStep));
img.push(shtFlm2pixel(bVals, shtYlm[i], (j + 0.5) * lonStep));
img.push(255);
}
}
return new ImageData(new Uint8ClampedArray(img), 64, 32);
}
}

// Vertex shader for equirectangular and cube


+ 2
- 1
utils/multires/Dockerfile View File

@@ -5,8 +5,9 @@ FROM ubuntu:20.04

ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 python3-dev python3-pil hugin-tools \
python3 python3-dev python3-numpy python3-pip python3-pil hugin-tools \
&& rm -rf /var/lib/apt/lists/*
RUN pip3 install pyshtools

ADD generate.py /generate.py
ENTRYPOINT ["python3", "/generate.py"]

+ 80
- 2
utils/multires/generate.py View File

@@ -1,7 +1,8 @@
#!/usr/bin/env python3

# Requires Python 3.2+ (or Python 2.7), the Python Pillow package,
# and nona (from Hugin)
# Requires Python 3.2+, the Python Pillow and NumPy packages, and
# nona (from Hugin). The Python pyshtools package is also needed for creating
# spherical-harmonic-transform previews (which are recommended).

# generate.py - A multires tile set generator for Pannellum
# Extensions to cylindrical input and partial panoramas by David von Oheimb
@@ -35,6 +36,9 @@ import math
import ast
from distutils.spawn import find_executable
import subprocess
import base64
import io
import numpy as np

# Allow large images (this could lead to a denial of service attack if you're
# running this script on user-submitted images.)
@@ -47,6 +51,57 @@ except KeyError:
# Handle case of PATH not being set
nona = None


genPreview = False
try:
import pyshtools as pysh
genPreview = True
except:
sys.stderr.write("Unable to import pyshtools. Not generating SHT preview.\n")

def img2shtHash(img, lmax=5):
'''
Create spherical harmonic transform (SHT) hash preview.
'''
def encodeFloat(f, maxVal):
return np.maximum(0, np.minimum(2 * maxVal, np.round(np.sign(f) * np.sqrt(np.abs(f)) * maxVal + maxVal))).astype(int)

def encodeCoeff(r, g, b, maxVal):
quantR = encodeFloat(r / maxVal, 9)
quantG = encodeFloat(g / maxVal, 9)
quantB = encodeFloat(b / maxVal, 9)
return quantR * 19 ** 2 + quantG * 19 + quantB

b83chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz#$%*+,-.:;=?@[]^_{|}~"

def b83encode(vals, length):
result = ""
for val in vals:
for i in range(1, length + 1):
result += b83chars[int(val // (83 ** (length - i))) % 83]
return result

# Calculate SHT coefficients
r = pysh.expand.SHExpandDH(img[..., 0], sampling=2, lmax_calc=lmax)
g = pysh.expand.SHExpandDH(img[..., 1], sampling=2, lmax_calc=lmax)
b = pysh.expand.SHExpandDH(img[..., 2], sampling=2, lmax_calc=lmax)

# Remove values above diagonal for both sine and cosine components
# Also remove first row and column for sine component
# These values are always zero
r = np.append(r[0][np.tril_indices(lmax + 1)], r[1, 1:, 1:][np.tril_indices(lmax)])
g = np.append(g[0][np.tril_indices(lmax + 1)], g[1, 1:, 1:][np.tril_indices(lmax)])
b = np.append(b[0][np.tril_indices(lmax + 1)], b[1, 1:, 1:][np.tril_indices(lmax)])

# Encode as string
maxVal = np.max([np.max(r), np.max(b), np.max(g)])
vals = encodeCoeff(r, g, b, maxVal).flatten()
asstr = b83encode(vals, 2)
lmaxStr = b83encode([lmax], 1)
maxValStr = b83encode(encodeFloat([2 * maxVal / 255 - 1], 41), 1)
return lmaxStr + maxValStr + asstr


# Subclass parser to add explaination for semi-option nona flag
class GenParser(argparse.ArgumentParser):
def error(self, message):
@@ -90,6 +145,8 @@ parser.add_argument('-q', '--quality', dest='quality', default=75, type=int,
help='output JPEG quality 0-100')
parser.add_argument('--png', action='store_true',
help='output PNG tiles instead of JPEG tiles')
parser.add_argument('--thumbnailsize', dest='thumbnailSize', default=0, type=int,
help='width of equirectangular thumbnail preview (defaults to no thumbnail; >512 not recommended)')
parser.add_argument('-n', '--nona', default=nona, required=nona is None,
metavar='EXECUTABLE',
help='location of the nona executable to use')
@@ -99,6 +156,7 @@ parser.add_argument('-d', '--debug', action='store_true',
help='debug mode (print status info and keep intermediate files)')
args = parser.parse_args()


# Create output directory
if os.path.exists(args.output):
print('Output directory "' + args.output + '" already exists')
@@ -229,6 +287,22 @@ if not args.debug:
if os.path.exists(os.path.join(args.output, face)):
os.remove(os.path.join(args.output, face))

# Generate preview (but not for partial panoramas)
if haov < 360 or vaov < 180:
genPreview = False
if genPreview:
# Generate SHT-hash preview
shtHash = img2shtHash(np.array(Image.open(args.inputFile)))
if args.thumbnailSize > 0:
# Create low-resolution base64-encoded equirectangular preview image
img = Image.open(args.inputFile)
img = img.resize((args.thumbnailSize, args.thumbnailSize // 2))
buf = io.BytesIO()
img.save(buf, format='JPEG', quality=75, optimize=True)
equiPreview = bytes('data:image/jpeg;base64,', encoding='utf-8')
equiPreview += base64.b64encode(buf.getvalue())
equiPreview = equiPreview.decode()

# Generate config file
text = []
text.append('{')
@@ -252,6 +326,10 @@ if args.autoload:
text.append(' "autoLoad": true,')
text.append(' "type": "multires",')
text.append(' "multiRes": {')
if genPreview:
text.append(' "shtHash": "' + shtHash + '",')
if args.thumbnailSize > 0:
text.append(' "equirectangularThumbnail": "' + equiPreview + '",')
text.append(' "path": "/%l/%s%y_%x",')
text.append(' "fallbackPath": "/fallback/%s",')
text.append(' "extension": "' + extension[1:] + '",')


+ 6
- 3
utils/multires/readme.md View File

@@ -16,11 +16,14 @@ or [Docker](https://www.docker.com/) can be used to avoid this installation.
### Option 1: with local dependencies

The `generate.py` script depends on `nona` (from [Hugin](http://hugin.sourceforge.net/)),
as well as Python with the [Pillow](https://pillow.readthedocs.org/) package. On Ubuntu,
these dependencies can be installed by running:
as well as Python 3 with the [Pillow](https://pillow.readthedocs.org/) and
[NumPy](https://numpy.org/) packages. The [pyshtools](https://shtools.github.io/SHTOOLS/)
Python package is also recommended. On Ubuntu, these dependencies can be
installed by running:

```bash
$ sudo apt install python3 python3-pil hugin-tools
$ sudo apt install python3 python3-pil python3-numpy python3-pip hugin-tools
$ pip3 install --user pyshtools
```

Once the dependencies are installed, a tileset can generated with:


Write Preview
Loading…
Cancel
Save