Search

We examine a logical foundation of depicting a Lorentz contraction of a Coulomb field (an electric field of a point charge in uniform motion) by means of the 'Lorentz contracted' field lines. Two existing arguments for a contraction of field lines sound appealing and lead to very simple calculations yielding the correct results. However, one of them is a victim to subtle logical weaknesses, as it relies on ascribing a degree of physical reality to the electric field lines. The other one correctly proves what it sets out to prove. But it does not provide a proof, or even a suggestion, of an additional result that can be obtained by a new poof that we present here. Though our idea is very simple, the calculations used to prove it - based on a little known, half a century old result by Tsien - are somewhat more involved than those from past arguments., Comment: 10 pages, 4 figures

We prove several inequalities between the curvature invariants, which impose constraints on curvature singularities, as well as asymptotic properties of spacetimes. Some of the inequalities hold for a family of spacetimes which include static, Friedmann--Lema\^itre--Robertson--Walker, and Bianchi type I metrics, independently of whether they are solutions of some particular field equations. In contrast, others hold for solutions of Einstein's gravitational field equation and a family of energy-momentum tensors, independently of the specific form of the spacetime metric. We illustrate different behaviour of the basic curvature invariants with numerous examples and discuss the consequences and limitations of the proven results., Comment: 26 pages

During the era of NASA's Apollo missions Keith S.~Runcorn proposed an explanation of discrepancy between the Moon's negligible global magnetic field and magnetised samples of lunar regolith, based on identical vanishing of external magnetic field of a spherical shell, magnetized by an internal source which is no longer present. We revisit and generalize the Runcorn's result, showing that it is a consequence of a (weighted) orthogonality of gradients of harmonic functions on a spherical shell in arbitrary number of dimensions. Furthermore, we explore bounds on external magnetic field in the case when the idealized spherical shell is replaced with a more realistic geometric shape and when the thermoremanent magnetization susceptibility deviates from the spherical symmetry. Finally, we analyse a model of thermoremanent magnetization acquired by crustal inward cooling of a spherical astrophysical body and put some general bounds on the associated magnetic field., Comment: 11 pages, 1 figure; comments are welcome!

We give a short introduction to the formalism of noncommutative (twisted) differential geometry that is used to derive the equations of motion for the gravitational perturbation of the Schwarzschild black hole in quantized spacetime. Special attention is given to quantum spacetime arising from $r - \varphi$ noncommutativity. Tortoise coordinate and near-horizon regions of the effective potentials are analyzed for both polar and axial modes. By carefully examining the associated Schr\"odinger-type equations, we provide the asymptotic solutions at the horizon and illustrate some differences between the polar and axial modes. These findings give further insight into the polar-axial isospectrality violation in the presence of the quantum structure of spacetime., Comment: 8 pages, 1 figure; contribution to "The XXVIII International Conference on Integrable Systems and Quantum Symmetries (ISQS28)", to be published in J.Phys.Conf.Ser

We study gravitational perturbations of the Schwarzschild metric in the context of noncommutative gravity. $r-\varphi$ and $r-t$ noncommutativity are introduced through a Moyal twist of the Hopf algebra of diffeomorphisms. Differential geometric structures such as curvature tensors are also twisted. Noncommutative equations of motion are derived from the recently proposed NC vacuum Einstein equation. Here, in addition to previously calculated axial NC potential, we present the polar solution which generalizes the work done by Zerilli. Quasinormal mode frequencies of the two potentials are calculated using three methods: WKB, P\"oschl-Teller and Rosen-Morse. Notably, we apply the WKB method up to the 13th order and determine the optimal order for each noncommutative parameter value individually. Additionally, we provide comprehensive error estimations for the higher-order WKB calculations, offering insights into the accuracy of our results. By comparing the spectra, we conclude that the classical isospectrality of axial and polar modes is broken upon spacetime quantization. Isospectrality is restored in the eikonal limit., Comment: 38 pages, 6 figures, 23 tables; typos corrected

Within the framework of noncommutative (NC) deformation of gauge field theory by the angular twist, we first rederive the NC scalar and gauge field model from our previous papers and then generalize it to the second order in the Seiberg-Witten (SW) map. It turns out that SW expansion is finite and that it ceases at the second order in the deformation parameter, ultimately giving rise to the equation of motion for the scalar field in Reissner--Nordstr\"om (RN) metric that is nonperturbative and exact at the same order. As a further step, we show that the effective metric put forth and constructed in our previous work satisfies the equations of Einstein-Maxwell gravity, but only within the first order of deformation and when the gauge field is fixed by the Coulomb potential of the charged black hole. Thus obtained NC deformation of the Reissner--Nordstr\"om (RN) metric appears to have an additional off-diagonal element which scales linearly with a deformation parameter. We analyze various properties of this metric., Comment: 16 pages, improved version

In classical Maxwell's electromagnetism, monopole term of the electric field is proportional to $r^{-2}$, while higher order multipole terms, sourced by anisotropic sources, fall-off faster. However, in nonlinear electromagnetism even a spherically symmetric field has multipole-like contributions. We prove that the leading subdominant term of the electric field, defined by nonlinear electromagnetic Lagrangian obeying Maxwellian weak field limit, in a static, spherically symmetric, asymptotically flat spacetime, is of the order $O(r^{-6})$ as $r \to \infty$. Moreover, using Lagrange inversion theorem and Fa\`a di Bruno's formula, we derive the series expansion of the electric field from the Taylor series of an analytic electromagnetic Lagrangian., Comment: 9 pages (published version; one footnote added and one sentence expanded)

Nonlinear extensions of classical Maxwell's electromagnetism are among the prominent candidates for theories admitting regular black hole solutions. A quest for such examples has been fruitful, but mostly unsystematic and littered by the introduction of physically unrealistic Lagrangians. We provide a procedure which admits the reconstruction of a nonlinear electromagnetic Lagrangian, consistent with the Euler--Heisenberg Lagrangian in the weak-field limit, from a given metric representing a regular, magnetically charged black hole., Comment: 8 pages; ver.2: one sentence added, version accepted for publication in Physics Letters B

In light of a recent direct experimental confirmation of a Lorentz contraction of Coulomb field (an electric field of a point charge in a uniform motion), we revisit some common confusions related to it, to be mindful of in teaching the subject. These include the questions about a radial nature of the field, a role of the retardation effect due to a finite speed of information transfer and some issues related to a depiction of Coulomb field by means of the Lorentz contracted field lines., Comment: 10 pages, 5 figures

We use the framework of Hopf algebra and noncommutative differential geometry to build a noncommutative (NC) theory of gravity in a bottom-up approach. Noncommutativity is introduced via deformed Hopf algebra of diffeomorphisms by means of a Drinfeld twist. The final result of the construction is a general formalism for obtaining NC corrections to the classical theory of gravity for a wide class of deformations and a general background. This also includes a novel proposal for noncommutative Einstein manifold. Moreover, the general construction is applied to the case of a linearized gravitational perturbation theory to describe a NC deformation of the metric perturbations. We specifically present an example for the Schwarzschild background and axial perturbations, which gives rise to a generalization of the work by Regge and Wheeler. All calculations are performed up to first order in perturbation of the metric and noncommutativity parameter. The main result is the noncommutative Regge-Wheeler potential. Finally, we comment on some differences in properties between the Regge-Wheeler potential and its noncommutative counterpart., Comment: 23 pages, 3 figures, typos corrected, published in JHEP

A quest for phenomenological footprints of quantum gravity is among the central scientific tasks in the rising era of gravitational wave astronomy. We study gravitational wave dynamics within the noncommutative geometry framework, based on a Drinfeld twist and newly proposed noncommutative Einstein equation, and obtain the leading quantum correction to Regge-Wheeler potential up to first order in the noncommutativity parameter. By calculating the quasinormal mode frequencies we show that the noncommutative Schwarzschild black hole remains stable under axial gravitational perturbations., Comment: 7 pages, 2 figures, improved version

GAN inversion aims at inverting given images into corresponding latent codes for Generative Adversarial Networks (GANs), especially StyleGAN where exists a disentangled latent space that allows attribute-based image manipulation at latent level. As most inversion methods build upon Convolutional Neural Networks (CNNs), we transfer a hierarchical vision Transformer backbone innovatively to predict $\mathcal{W^+}$ latent codes at token level. We further apply a Style-driven Multi-scale Adaptive Refinement Transformer (SMART) in $\mathcal{F}$ space to refine the intermediate style features of the generator. By treating style features as queries to retrieve lost identity information from the encoder's feature maps, SMART can not only produce high-quality inverted images but also surprisingly adapt to editing tasks. We then prove that StylePrompter lies in a more disentangled $\mathcal{W^+}$ and show the controllability of SMART. Finally, quantitative and qualitative experiments demonstrate that StylePrompter can achieve desirable performance in balancing reconstruction quality and editability, and is "smart" enough to fit into most edits, outperforming other $\mathcal{F}$-involved inversion methods., Comment: Some figures in the appendix are compressed for the reason of arXiv submission constrict

Image quality assessment is a fundamental problem in the field of image processing, and due to the lack of reference images in most practical scenarios, no-reference image quality assessment (NR-IQA), has gained increasing attention recently. With the development of deep learning technology, many deep neural network-based NR-IQA methods have been developed, which try to learn the image quality based on the understanding of database information. Currently, Transformer has achieved remarkable progress in various vision tasks. Since the characteristics of the attention mechanism in Transformer fit the global perceptual impact of artifacts perceived by a human, Transformer is thus well suited for image quality assessment tasks. In this paper, we propose a Transformer based NR-IQA model using a predicted objective error map and perceptual quality token. Specifically, we firstly generate the predicted error map by pre-training one model consisting of a Transformer encoder and decoder, in which the objective difference between the distorted and the reference images is used as supervision. Then, we freeze the parameters of the pre-trained model and design another branch using the vision Transformer to extract the perceptual quality token for feature fusion with the predicted error map. Finally, the fused features are regressed to the final image quality score. Extensive experiments have shown that our proposed method outperforms the current state-of-the-art in both authentic and synthetic image databases. Moreover, the attentional map extracted by the perceptual quality token also does conform to the characteristics of the human visual system., Comment: Submitted to TMM

Discovery that gravitational field equations may coerce the spacetime metric with isometries to attain a block-diagonal form compatible with these isometries, was one of the gems built into the corpus of black hole uniqueness theorems. We revisit the geometric background of a block-diagonal metric with isometries, foliation defined by Killing vector fields and the corresponding Godbillon-Vey characteristic class. Furthermore, we analyse sufficient conditions for various matter sources, including scalar, nonlinear electromagnetic and Proca fields, that imply the isometry-compatible block-diagonal form of the metric. Finally, we generalize the theorem on the absence of null electromagnetic fields in static spacetimes to an arbitrary number of spacetime dimensions, wide class of gravitational field equations and nonlinear electromagnetic fields., Comment: 22 pages (ver. 3: equation (51) corrected to m-dimensional case)

Point clouds have become increasingly prevalent in representing 3D scenes within virtual environments, alongside 3D meshes. Their ease of capture has facilitated a wide array of applications on mobile devices, from smartphones to autonomous vehicles. Notably, point cloud compression has reached an advanced stage and has been standardized. However, the availability of quality assessment datasets, which are essential for developing improved objective quality metrics, remains limited. In this paper, we introduce BASICS, a large-scale quality assessment dataset tailored for static point clouds. The BASICS dataset comprises 75 unique point clouds, each compressed with four different algorithms including a learning-based method, resulting in the evaluation of nearly 1500 point clouds by 3500 unique participants. Furthermore, we conduct a comprehensive analysis of the gathered data, benchmark existing point cloud quality assessment metrics and identify their limitations. By publicly releasing the BASICS dataset, we lay the foundation for addressing these limitations and fostering the development of more precise quality metrics., Comment: Published in IEEE TMM, 14 pages, 8 figures

We study the noncommutative corrections to the entropy of the Reissner-Nordstr\"{o}m black hole using a $\kappa$-deformed scalar probe within the brick-wall framework. The noncommutativity is encoded in an Abelian Drinfeld twist constructed from the Killing vector fields of the Reissner-Nordstr\"{o}m black hole. We show that the noncommutative effects naturally lead to a logarithmic correction to the Bekenstein-Hawking entropy even at the lowest order of the WKB approximation. In contrast, such logarithmic corrections in the commutative setup appear only after the quantum effects are included through higher order WKB corrections or through higher loop effects. Our analysis thus provides further evidence towards the hypothesis that the noncommutative framework is capable of encoding quantum effects in curved spacetime., Comment: 14 pages, slightly differs from the published JHEP version

Compression is an important task for many practical applications of light fields. Although previous work has proposed numerous methods for efficient light field compression, the effect of view selection on this task is not well exploited. In this work, we study different sub-sampling and reconstruction strategies for light field compression. We apply various sub-sampling and corresponding reconstruction strategies before and after light field compression. Then, fully reconstructed light fields are assessed to evaluate the performance of different methods. Our evaluation is performed on both real-world and synthetic datasets, and optimal strategies are devised from our experimental results. We hope this study would be beneficial for future research such as light field streaming, storage, and transmission.

Monocular 3D human pose estimation technologies have the potential to greatly increase the availability of human movement data. The best-performing models for single-image 2D-3D lifting use graph convolutional networks (GCNs) that typically require some manual input to define the relationships between different body joints. We propose a novel transformer-based approach that uses the more generalised self-attention mechanism to learn these relationships within a sequence of tokens representing joints. We find that the use of intermediate supervision, as well as residual connections between the stacked encoders benefits performance. We also suggest that using error prediction as part of a multi-task learning framework improves performance by allowing the network to compensate for its confidence level. We perform extensive ablation studies to show that each of our contributions increases performance. Furthermore, we show that our approach outperforms the recent state of the art for single-frame 3D human pose estimation by a large margin. Our code and trained models are made publicly available on Github., Comment: Accepted to ICPR 2022

Computer vision/deep learning-based 3D human pose estimation methods aim to localize human joints from images and videos. Pose representation is normally limited to 3D joint positional/translational degrees of freedom (3DOFs), however, a further three rotational DOFs (6DOFs) are required for many potential biomechanical applications. Positional DOFs are insufficient to analytically solve for joint rotational DOFs in a 3D human skeletal model. Therefore, we propose a temporal inverse kinematics (IK) optimization technique to infer joint orientations throughout a biomechanically informed, and subject-specific kinematic chain. For this, we prescribe link directions from a position-based 3D pose estimate. Sequential least squares quadratic programming is used to solve a minimization problem that involves both frame-based pose terms, and a temporal term. The solution space is constrained using joint DOFs, and ranges of motion (ROMs). We generate 3D pose motion sequences to assess the IK approach both for general accuracy, and accuracy in boundary cases. Our temporal algorithm achieves 6DOF pose estimates with low Mean Per Joint Angular Separation (MPJAS) errors (3.7{\deg}/joint overall, & 1.6{\deg}/joint for lower limbs). With frame-by-frame IK we obtain low errors in the case of bent elbows and knees, however, motion sequences with phases of extended/straight limbs results in ambiguity in twist angle. With temporal IK, we reduce ambiguity for these poses, resulting in lower average errors., Comment: Project page: https://kevgildea.github.io/KinePose/

Aspect ratio and spatial layout are two of the principal factors determining the aesthetic value of a photograph. But, incorporating these into the traditional convolution-based frameworks for the task of image aesthetics assessment is problematic. The aspect ratio of the photographs gets distorted while they are resized/cropped to a fixed dimension to facilitate training batch sampling. On the other hand, the convolutional filters process information locally and are limited in their ability to model the global spatial layout of a photograph. In this work, we present a two-stage framework based on graph neural networks and address both these problems jointly. First, we propose a feature-graph representation in which the input image is modelled as a graph, maintaining its original aspect ratio and resolution. Second, we propose a graph neural network architecture that takes this feature-graph and captures the semantic relationship between the different regions of the input image using visual attention. Our experiments show that the proposed framework advances the state-of-the-art results in aesthetic score regression on the Aesthetic Visual Analysis (AVA) benchmark., Comment: International Conference on Pattern Recognition (ICPR), 2022

One of the long standing problems is a quest for regular black hole solutions, in which a resolution of the spacetime singularity has been achieved by some physically reasonable, classical field, before one resorts to the quantum gravity. The prospect of using nonlinear electromagnetic fields for this goal has been limited by the Bronnikov's no-go theorems, focused on Lagrangians depending on the electromagnetic invariant $F_{ab}F^{ab}$ only. We extend Bronnikov's results by taking into account Lagrangians that depend on both electromagnetic invariants, $F_{ab}F^{ab}$ and $F_{ab}\,{\star F^{ab}}$, and prove that the tension between the Lagrangian's Maxwellian weak field limit and boundedness of the curvature invariants persists in more general class of theories., Comment: 10 pages (two columns); ver.2: several references added; ver. 3: version accepted for publication in Physical Review D

We prove two theorems which imply that any stationary nonlinear electromagnetic field obeying a dominant energy condition in a strictly stationary, everywhere regular, asymptotically flat spacetime must be either trivial or a stealth field. The first theorem holds in static spacetimes and is independent of the gravitational part of the action, as long as the coupling of the electromagnetic field to the gravitational field is minimal. The second theorem assumes Einstein--Hilbert gravitational action and relies on the positive energy theorem, but does not assume that the spacetime metric is static. In addition, we discuss possible generalizations of these results, to theories with charged matter, as well as higher-dimensional nonlinear electromagnetic fields., Comment: 8.1 pages (two columns); ver.2: version accepted for publication in Physical Review D (several references added and some typos corrected)

Most of existing video action recognition models ingest raw RGB frames. However, the raw video stream requires enormous storage and contains significant temporal redundancy. Video compression (e.g., H.264, MPEG-4) reduces superfluous information by representing the raw video stream using the concept of Group of Pictures (GOP). Each GOP is composed of the first I-frame (aka RGB image) followed by a number of P-frames, represented by motion vectors and residuals, which can be regarded and used as pre-extracted features. In this work, we 1) introduce sampling the input for the network from partially decoded videos based on the GOP-level, and 2) propose a plug-and-play mulTi-modal lEArning Module (TEAM) for training the network using information from I-frames and P-frames in an end-to-end manner. We demonstrate the superior performance of TEAM-Net compared to the baseline using RGB only. TEAM-Net also achieves the state-of-the-art performance in the area of video action recognition with partial decoding. Code is provided at https://github.com/villawang/TEAM-Net., Comment: To appear in BMVC 2021

In this paper, we study the spectral properties of re-parameterized light field. Following previous studies of the light field spectrum, which notably provided sampling guidelines, we focus on the two plane parameterization of the light field. However, we introduce additional flexibility by allowing the image plane to be tilted and not only parallel. A formal theoretical analysis is first presented, which shows that more flexible sampling guidelines (i.e. wider camera baselines) can be used to sample the light field when adapting the image plane orientation to the scene geometry. We then present our simulations and results to support these theoretical findings. While the work introduced in this paper is mostly theoretical, we believe these new findings open exciting avenues for more practical application of light fields, such as view synthesis or compact representation.

We address the problem of exposure correction of dark, blurry and noisy images captured in low-light conditions in the wild. Classical image-denoising filters work well in the frequency space but are constrained by several factors such as the correct choice of thresholds, frequency estimates etc. On the other hand, traditional deep networks are trained end-to-end in the RGB space by formulating this task as an image-translation problem. However, that is done without any explicit constraints on the inherent noise of the dark images and thus produce noisy and blurry outputs. To this end we propose a DCT/FFT based multi-scale loss function, which when combined with traditional losses, trains a network to translate the important features for visually pleasing output. Our loss function is end-to-end differentiable, scale-agnostic, and generic; i.e., it can be applied to both RAW and JPEG images in most existing frameworks without additional overhead. Using this loss function, we report significant improvements over the state-of-the-art using quantitative metrics and subjective tests., Comment: Published version 8 pages. SIViP (2021)

In natural image matting, the goal is to estimate the opacity of the foreground object in the image. This opacity controls the way the foreground and background is blended in transparent regions. In recent years, advances in deep learning have led to many natural image matting algorithms that have achieved outstanding performance in a fully automatic manner. However, most of these algorithms only predict the alpha matte from the image, which is not sufficient to create high-quality compositions. Further, it is not possible to manually interact with these algorithms in any way except by directly changing their input or output. We propose a novel recurrent neural network that can be used as a post-processing method to recover the foreground and background colors of an image, given an initial alpha estimation. Our method outperforms the state-of-the-art in color estimation for natural image matting and show that the recurrent nature of our method allows users to easily change candidate solutions that lead to superior color estimations., Comment: To be published in WACV 2021

Spatial-temporal, channel-wise, and motion patterns are three complementary and crucial types of information for video action recognition. Conventional 2D CNNs are computationally cheap but cannot catch temporal relationships; 3D CNNs can achieve good performance but are computationally intensive. In this work, we tackle this dilemma by designing a generic and effective module that can be embedded into 2D CNNs. To this end, we propose a spAtio-temporal, Channel and moTion excitatION (ACTION) module consisting of three paths: Spatio-Temporal Excitation (STE) path, Channel Excitation (CE) path, and Motion Excitation (ME) path. The STE path employs one channel 3D convolution to characterize spatio-temporal representation. The CE path adaptively recalibrates channel-wise feature responses by explicitly modeling interdependencies between channels in terms of the temporal aspect. The ME path calculates feature-level temporal differences, which is then utilized to excite motion-sensitive channels. We equip 2D CNNs with the proposed ACTION module to form a simple yet effective ACTION-Net with very limited extra computational cost. ACTION-Net is demonstrated by consistently outperforming 2D CNN counterparts on three backbones (i.e., ResNet-50, MobileNet V2 and BNInception) employing three datasets (i.e., Something-Something V2, Jester, and EgoGesture). Codes are available at \url{https://github.com/V-Sense/ACTION-Net}., Comment: To appear in CVPR 2021

As the interaction between the black holes and highly energetic infalling charged matter receives quantum corrections, the basic laws of black hole mechanics have to be carefully rederived. Using the covariant phase space formalism, we generalize the first law of black hole mechanics, both "equilibrium state" and "physical process" versions, in the presence of nonlinear electrodynamics fields, defined by Lagrangians depending on both quadratic electromagnetic invariants, $F_{ab}F^{ab}$ and $F_{ab}\,{\star F}^{ab}$. Derivation of this law demands a specific treatment of the Lagrangian parameters, similar to embedding of the cosmological constant into thermodynamic context. Furthermore, we discuss the validity of energy conditions, several complementing proofs of the zeroth law of black hole electrodynamics and some aspects of the recently generalized Smarr formula, its (non-)linearity and relation to the first law., Comment: 20 pages (two columns), 2 figures; ver.3: version accepted for publication in Physical Review D (several additional technical comments and references; one paragraph added to the "Discussion")

In 1981 Wyman classified the solutions of the Einstein--Klein--Gordon equations with static spherically symmetric spacetime metric and vanishing scalar potential. For one of these classes, the scalar field linearly grows with time. We generalize this symmetry noninheriting solution, perturbatively, to a rotating one and extend the static solution exactly to arbitrary spacetime dimensions. Furthermore, we investigate the existence of nonminimally coupled, time-dependent real scalar fields on top of static black holes, and prove a no-hair theorem for stealth scalar fields on the Schwarzschild background., Comment: 17 pages, 1 figure; minor corrections, more refs, accepted in CQG

DuctTake is a system designed to enable practical compositing of multiple takes of a scene into a single video. Current industry solutions are based around object segmentation, a hard problem that requires extensive manual input and cleanup, making compositing an expensive part of the film-making process. Our method instead composites shots together by finding optimal spatiotemporal seams using motion-compensated 3D graph cuts through the video volume. We describe in detail the required components, decisions, and new techniques that together make a usable, interactive tool for compositing HD video, paying special attention to running time and performance of each section. We validate our approach by presenting a wide variety of examples and by comparing result quality and creation time to composites made by professional artists using current state-of-the-art tools.

High angular resolution is advantageous for practical applications of light fields. In order to enhance the angular resolution of light fields, view synthesis methods can be utilized to generate dense intermediate views from sparse light field input. Most successful view synthesis methods are learning-based approaches which require a large amount of training data paired with ground truth. However, collecting such large datasets for light fields is challenging compared to natural images or videos. To tackle this problem, we propose a self-supervised light field view synthesis framework with cycle consistency. The proposed method aims to transfer prior knowledge learned from high quality natural video datasets to the light field view synthesis task, which reduces the need for labeled light field data. A cycle consistency constraint is used to build bidirectional mapping enforcing the generated views to be consistent with the input views. Derived from this key concept, two loss functions, cycle loss and reconstruction loss, are used to fine-tune the pre-trained model of a state-of-the-art video interpolation method. The proposed method is evaluated on various datasets to validate its robustness, and results show it not only achieves competitive performance compared to supervised fine-tuning, but also outperforms state-of-the-art light field view synthesis methods, especially when generating multiple intermediate views. Besides, our generic light field view synthesis framework can be adopted to any pre-trained model for advanced video interpolation., Comment: Accepted at MMSP 2020

Limited angular resolution is one of the main obstacles for practical applications of light fields. Although numerous approaches have been proposed to enhance angular resolution, view selection strategies have not been well explored in this area. In this paper, we study subsampling and reconstruction strategies for light fields. First, different subsampling strategies are studied with a fixed sampling ratio, such as row-wise sampling, column-wise sampling, or their combinations. Second, several strategies are explored to reconstruct intermediate views from four regularly sampled input views. The influence of the angular density of the input is also evaluated. We evaluate these strategies on both real-world and synthetic datasets, and optimal selection strategies are devised from our results. These can be applied in future light field research such as compression, angular super-resolution, and design of camera systems., Comment: Accepted at IMVIP 2020

Depth map estimation is a crucial task in computer vision, and new approaches have recently emerged taking advantage of light fields, as this new imaging modality captures much more information about the angular direction of light rays compared to common approaches based on stereoscopic images or multi-view. In this paper, we propose a novel depth estimation method from light fields based on existing optical flow estimation methods. The optical flow estimator is applied on a sequence of images taken along an angular dimension of the light field, which produces several disparity map estimates. Considering both accuracy and efficiency, we choose the feature flow method as our optical flow estimator. Thanks to its spatio-temporal edge-aware filtering properties, the different disparity map estimates that we obtain are very consistent, which allows a fast and simple aggregation step to create a single disparity map, which can then converted into a depth map. Since the disparity map estimates are consistent, we can also create a depth map from each disparity estimate, and then aggregate the different depth maps in the 3D space to create a single dense depth map., Comment: Accepted at IMVIP 2017

The effective design of visual computing systems depends heavily on the anticipation of visual attention, or saliency. While visual attention is well investigated for conventional 2D images and video, it is nevertheless a very active research area for emerging immersive media. In particular, visual attention of light fields (light rays of a scene captured by a grid of cameras or micro lenses) has only recently become a focus of research. As they may be rendered and consumed in various ways, a primary challenge that arises is the definition of what visual perception of light field content should be. In this work, we present a visual attention study on light field content. We conducted perception experiments displaying them to users in various ways and collected corresponding visual attention data. Our analysis highlights characteristics of user behaviour in light field imaging applications. The light field data set and attention data are provided with this paper., Comment: To appear in Irish Machine Vision and Image Processing (IMVIP) 2020

Convolutional neural network (CNN)-based methods have achieved great success for single-image superresolution (SISR). However, most models attempt to improve reconstruction accuracy while increasing the requirement of number of model parameters. To tackle this problem, in this paper, we study reducing the number of parameters and computational cost of CNN-based SISR methods while maintaining the accuracy of super-resolution reconstruction performance. To this end, we introduce a novel network architecture for SISR, which strikes a good trade-off between reconstruction quality and low computational complexity. Specifically, we propose an iterative back-projection architecture using sub-pixel convolution instead of deconvolution layers. We evaluate the performance of computational and reconstruction accuracy for our proposed model with extensive quantitative and qualitative evaluations. Experimental results reveal that our proposed method uses fewer parameters and reduces the computational cost while maintaining reconstruction accuracy against state-of-the-art SISR methods over well-known four SR benchmark datasets. Code is available at "https://github.com/supratikbanerjee/SubPixel-BackProjection_SuperResolution"., Comment: To appear in IMVIP 2020

The capacitance matrix relates potentials and charges on a system of conductors. We review and rigorously generalize its properties, block-diagonal structure and inequalities, deduced from the geometry of system of conductors and analytic properties of the permittivity tensor. Furthermore, we discuss alternative choices of regularization of the capacitance matrix, which allow us to find the charge exchanged between the conductors having been brought to an equal potential. Finally, we discuss the tacit approximations used in standard treatments of the electric circuits, demonstrating how the formulae for the capacitance of capacitors connected in parallel and series may be recovered from the capacitance matrix., Comment: ver. 2: 18 pages, 7 figures; minor revision, accepted for publication in PIER B

Multimedia and creativity software products are being used to edit and control various elements of creative media practices. These days, the technical affordances of mobile multimedia devices and the advent of high-speed 5G internet access mean that these abilities are simpler and more readily available to be harnessed by mobile applications. In this paper, using a prototype application, we discuss how potential users of such technology are motivated to use a video recoloring application and explore the role that user expectation and satisfaction play in this process. By exploring this topic and focusing on the human-computer interaction, we found that color transfer interactions are driven by several intrinsic motivations and that user expectations and satisfaction ratings can be maintained via clear visualizations of the processes to be undertaken. Furthermore, we reveal the specific language that users use to communicate video recoloring when regarding user motivations, expectations, and satisfaction. This research provides important information for developers of state-of-art recoloring processes and contributes to dialogues surrounding the users of mobile multimedia technology in practice.

Egocentric gestures are the most natural form of communication for humans to interact with wearable devices such as VR/AR helmets and glasses. A major issue in such scenarios for real-world applications is that may easily become necessary to add new gestures to the system e.g., a proper VR system should allow users to customize gestures incrementally. Traditional deep learning methods require storing all previous class samples in the system and training the model again from scratch by incorporating previous samples and new samples, which costs humongous memory and significantly increases computation over time. In this work, we demonstrate a lifelong 3D convolutional framework -- c(C)la(a)ss increment(t)al net(Net)work (CatNet), which considers temporal information in videos and enables lifelong learning for egocentric gesture video recognition by learning the feature representation of an exemplar set selected from previous class samples. Importantly, we propose a two-stream CatNet, which deploys RGB and depth modalities to train two separate networks. We evaluate CatNets on a publicly available dataset -- EgoGesture dataset, and show that CatNets can learn many classes incrementally over a long period of time. Results also demonstrate that the two-stream architecture achieves the best performance on both joint training and class incremental training compared to 3 other one-stream architectures. The codes and pre-trained models used in this work are provided at https://github.com/villawang/CatNet., Comment: CVPR 2020 Workshop at Continual Learning (CLVISION)