Evaluation of video quality through compact energy representation in the 3D DCT domain (2023)

In recent years, deep learning has attracted significant research attention for video recovery. Among the existing contributions, the single-frame based approaches rely purely on a frame of reference and neglect the remaining neighboring frames when improving a target frame. In contrast, the multi-frame based contributions use temporal information in a sliding window, and the existing recurring design uses only a single preceding extended frame. It is intuitive to use both multiple original adjacent frames and the preceding enhanced frames to improve video quality. In this article, we propose a Recurrent Video Quality Enhancement Framework using Multi-Granularity Frame Fusion and Frame Difference Based Attention (REMD). First, we develop a three-dimensional convolutional neural network-based encoder-decoder fusion model that fuses multiple frames in multigranularity. Second, severe compression artifacts tend to show up around the edges and textures of the compressed frames. We propose a frame-difference-based spatial attention method to intensify the edges and textures of moving regions. Finally, a repetitive sliding window design is conceived to exploit the temporal information in preceding enhanced frames and subsequent neighboring frames. Experiments show that our method achieves superior performance compared to the prior art contributions with significantly reduced spatial and computational complexity.

Because the marginal distribution of the wavelet transform output is a strong-tailed bell-shaped function characterized by a larger fraction of small wavelet coefficients or even zeros, the traditional feature extraction approaches such as wavelet energy spectrum and energy spectrum entropy are used in bearing error detection applications cannot accurately express the statistical characteristic of wavelet subbands. In this paper, we propose a novel wavelet-based bearing error detection approach using wavelet transform and Generalized Gaussian Density (GGD) modelling. A GGD-based feature descriptor is generated from the concatenation of the statistical parameters of each wavelet subband estimated by the maximum likelihood method. According to the descriptor information, a class label for bearing error detection is assigned by the subsequent classifier. The extensive experimental results show that the proposed approach can capture wavelet subband information of bearing vibration signals more accurately and flexibly than energy, energy entropy, Gaussian, and Laplacian-based feature description methods. Furthermore, the experimental results with different wavelet filters, decomposition levels and classifiers show that the new method significantly improves the error detection accuracy compared to conventional approaches with better robustness.

Video manipulation methods have made significant advances in recent years. This is partly due to the rapid development of advanced deep learning methods and also due to the large amount of video material that is now in the public domain. In the past, persuasive video manipulation was too labor intensive to achieve at scale. However, recent developments in methods based on deep learning have made it possible not only to produce convincing fake videos, but also to fully synthesize video content. Such advances offer new opportunities to enhance visual content itself, but at the same time pose new challenges for state-of-the-art methods for detecting tampering. Video tampering detection has been an active area of ​​research for some time, with regular reviews of the topic. However, little attention has been paid to video manipulation techniques themselves. This paper provides an objective and thorough examination of current techniques related to digital video manipulation. We thoroughly examine their evolution and show how current scoring techniques offer opportunities for the advancement of video tampering detection. A critical and comprehensive review of photorealistic video synthesis is provided with an emphasis on deep learning-based methods. Existing manipulated video datasets are also qualitatively checked and critically discussed. Finally, conclusions are drawn from a comprehensive and thorough review of manipulation methods with discussions of future research directions aimed at improving detection methods.

Visual quality assessment (VQA) methods applicable to assess noise, blur, and MPEG distortion can be viewed as an extension of picture quality assessment (IQA) methods. At the same time, the VQA is a special issue for studying in the sense of an available set of final frameworks. Given our interest in image blur assessment, we focus the discussion on the non-reference methods related to the main types of blur occurring in video sequences such as: B. Motion Blur, Defocus Blur, Shake Blur, and Atmospheric Blur. First, we detect the blurred areas in the frame using a simple imposed blur method. Second, a degree of quality of the blur is evaluated according to the defined type. The obtained estimators facilitate the initial level for blurring algorithms to decide whether to restore or remove the blurred frames from the following processing. The Sports Videos in the Wild (SVW) dataset provides the rich test material for experiments. The resulting estimators agree well with the human visual system (HVS) perceptual assessment.

Neurocomputing, Band 269, 2017, S. 21-28

The sparse representation of signals has become an important tool in computer vision. Sparse representations have achieved remarkable feats in many computer vision applications such as image denoising, image superresolution, and object detection. Sparse representation models often contain two phases: sparse coding and dictionary learning. In this article, we propose a nonlinear, nonnegative, sparse representation model: NNK-KSVD. In the sparse coding stage, a non-linear update rule is proposed to preserve the sparse matrix. In the dictionary learning phase, the proposed model extends the kernel KSVD by embedding the non-negative sparse encoding. The proposed non-negative kernel-sparse representation model was evaluated on several publicly available image datasets for the classification task. Experimental results show that by exploiting the nonlinear structure in images and exploiting the "additive" nature of non-negative sparse coding, promising classification performance can be achieved. In addition, the proposed sparse representation method was also evaluated in image research tasks, yielding competitive results.

Neurocomputing, Band 269, 2017, S. 212-219

Recently, the forum for questions and answers (Q&A) for software development (e.g. Stack Overflow) is becoming popular. Identifying the best answer to a question asked is important for the Q&A forum because the best answer that provides an excellent solution to the question asked can guide developers in solving their problems in practice. However, the best answers are often not explicitly marked by the question owners. For other developers with the same question, it would be time consuming to review all candidate answers to find the appropriate one. In this whitepaper, we propose a novel approach to predict the best answers to the questions asked on Stack Overflow using heterogeneous data sources in the forum. We extract different group functions from multiple data sources and combine them for the final prediction through multi-view learning. Experimental results show that the proposed method is effective for identifying the best answers to questions asked on Stack Overflow.

Neurocomputing, Band 269, 2017, S. 73-81

A language model is fundamental to many applications in natural language processing. Most language models are trained on a large set of datasets and are difficult to adapt to other areas where only a small dataset may be available. Optimizing discounting parameters for smoothing is one way to adapt language models for a new domain. In this work, we present novel language models based on tunable discounting mechanisms. The language models are trained on a large dataset, but their discounting parameters can then be tuned to a target dataset. We study tunable discounting and polynomial discounting functions based on the modified Kneser-Ney (mKN) models. In particular, we propose the tunable mKN (TmKN) model, the polynomial discounting mKN (PmKN) model, and the tunable and polynomial discounting mKN (TPmKN) model. We test our proposed models and compare them to the mKN model, the improved KN model, and the tunable mKN with the interpolation model (mKN + interp). With the implementation, our language models achieve perplexity improvements in both in-domain and out-of-domain evaluation. Experimental results show that our new models significantly outperform the base model and our models are particularly adept at adapting to new domains. In addition, we use the visualization technique to depict the relationship between parameter settings and the language model performances to guide our parameter optimization process. Exploratory visual analysis is then used to examine the performance of the proposed language models, revealing the strength and characteristics of the models.

Neurocomputing, Band 269, 2017, S. 117-119

Neurocomputing, Band 269, 2017, S. 82-89

In instance-based approaches, human pose estimation is achieved by retrieving relevant poses with images. Therefore, image description is crucial and it is common to extract multiple features to better describe the visual input data. To merge multiple features, traditional methods simply concatenate multi-view features into one long vector. This oversimplified process suffers from two shortcomings: (1) it usually results in long feature vectors that suffer from the "curse of dimensionality"; (2) it does not make physical sense and may not be able to fully exploit the complementary properties of multi-view features. To address such issues in this article, we present a dimensionality reduction method based on sparse spectral embedding followed by nearest neighbor regression ensemble in low-rank multi-view feature space to derive 3D human poses from monocular videos. The experiments on the HumanEva data set show the effectiveness of the proposed method.

Neurocomputing, Band 269, 2017, S. 1-2