FastWordBug A Fast Method To Generate Adversarial Text Against NLP Applications. In this paper, we present a novel algorithm, FastWordBug, to efficiently generate small text perturbations in a black box setting that forces a sentiment analysis or text classification mode to make an incorrect prediction. By combining the part of speech attributes of words, we propose a scoring method that can quickly identify important words that affect text classification. We evaluate FastWordBug on three real world text datasets and two state of the art machine learning models under black box setting. The results show that our method can significantly reduce the accuracy of the model, and at the same time, we can call the model as little as possible, with the highest attack efficiency. We also attack two popular real world cloud services of NLP, and the results show that our method works as well.


Atomic resolution imaging of carbonyl sulfide by laser induced electron diffraction. Measurements on the strong field ionization of carbonyl sulfide molecules by short, intense, 2 um wavelength laser pulses are presented from experiments where angle resolved photoelectron distributions were recorded with a high energy velocity map imaging spectrometer, designed to reach a maximum kinetic energy of 500 eV. The laser field free elastic scattering cross section of carbonyl sulfide was extracted from the measurements and is found in good agreement with previous experiments, performed using conventional electron diffraction. By comparing our measurements to the results of calculations, based on the quantitative rescattering theory (QRS), the bond lengths and molecular geometry were extracted from the experimental differential cross sections to a precision better than pm5 pm and in agreement with the known values.


On Generating Extended Summaries of Long Documents. Prior work in document summarization has mainly focused on generating short summaries of a document. While this type of summary helps get a high level view of a given document, it is desirable in some cases to know more detailed information about its salient points that can't fit in a short summary. This is typically the case for longer documents such as a research paper, legal document, or a book. In this paper, we present a new method for generating extended summaries of long papers. Our method exploits hierarchical structure of the documents and incorporates it into an extractive summarization model through a multi task learning approach. We then present our results on three long summarization datasets, arXiv Long, PubMed Long, and Longsumm. Our method outperforms or matches the performance of strong baselines. Furthermore, we perform a comprehensive analysis over the generated results, shedding insights on future research for long form summary generation task. Our analysis shows that our multi tasking approach can adjust extraction probability distribution to the favor of summary worthy sentences across diverse sections. Our datasets, and codes are publicly available at https github.com Georgetown IR Lab ExtendedSumm


Type II Finite time blow up for the three dimensional energy critical heat equation. We consider the following Cauchy problem for three dimensional energy critical heat equation begin equation begin cases u t Delta u u 5 , mbox in mathbb R 3 times (0,T), u(x,0) u 0(x), mbox in mathbb R 3. end cases end equation We construct type II finite time blow up solution u(x,t) with the blow up rates u L infty sim (T t) k , where k 1,2,... . This gives a rigorous proof of the formal computations by Filippas, Herrero and Velazquez cite fhv . This is the first instance of type II finite time blow up for three dimensional energy critical heat equation.


Concentration on Poisson spaces via modified Phi Sobolev inequalities. Concentration properties of functionals of general Poisson processes are studied. Using a modified Phi Sobolev inequality a recursion scheme for moments is established, which is of independent interest. This is applied to derive moment and concentration inequalities for functionals on abstract Poisson spaces. Applications of the general results in stochastic geometry, namely Poisson cylinder models and Poisson random polytopes, are presented as well.


An inequality for the number of periods in a word. We prove an inequality for the number of periods in a word x in terms of the length of x and its initial critical exponent. Next, we characterize all periods of the length n prefix of a characteristic Sturmian word in terms of the lazy Ostrowski representation of n, and use this result to show that our inequality is tight for infinitely many words x. We propose two related measures of periodicity for infinite words. Finally, we also consider special cases where x is overlap free or squarefree.


Role of vortical structures for enstrophy and scalar transport in flows with and without stable stratification. In this paper, we investigate the enstrophy dynamics in relation to objective Eulerian coherent structures (OECSs) and their impact on the enstrophy and scalar transport near the turbulent non turbulent interface (TNTI) in flows with and without stable stratification. We confirm that vortex stretching produces enstrophy inside the boundaries of the OECSs, while viscous diffusion transfers the enstrophy across the boundaries of the structures. Although often overlooked in the literature, viscous dissipation of enstrophy within the boundaries of vortical structures is significant. Conversely, for the weakly stratified flows also investigated here, the effect of the baroclinic torque is negligible. We provide evidence that the OECSs advect the passive active scalar and redistribute it via molecular diffusion. Finally, we use conditional analysis to show that the typical profiles of the enstrophy and scalar transport equation terms across the TNTI are compatible with the presence of OECSs positioned at the edge between the turbulent sublayer and the turbulent core region. We show that when these profiles are further conditioned to the presence of OECSs, their magnitude is considerably higher.


Erd H o s Ko Rado theorem for vector spaces over residue class rings. Let h prod i 1 t p i s i be its decomposition into a product of powers of distinct primes, and mathbb Z h be the residue class ring modulo h . Let mathbb Z h n be the n dimensional row vector space over mathbb Z h . A generalized Grassmann graph for mathbb Z h n , denoted by G r(m,n, mathbb Z h ) ( G r for short), has all m subspaces of mathbb Z h n as its vertices, and two distinct vertices are adjacent if their intersection is of dimension m r , where 2 leq r leq m 1 leq n . In this paper, we determine the clique number and geometric structures of maximum cliques of G r . As a result, we obtain the Erd H o s Ko Rado theorem for mathbb Z h n .


Leveraging Context awareness to Better Support the IoT Cloud Edge Continuum. Novel Internet of Things (IoT) requirements derived from a broader interconnection of heterogeneous devices have pushed the horizons of Cloud computing and are giving rise to a wider decentralisation of applications and data centers. An answer to the underlying network concerns, such as the need to lower the resulting latency due to heavy computation needs,or safety aspects, gave rise to Edge Fog computing, where IoT functionality can be also supported closer to data sources. While it is today feasible to perform some IoT functionality on the Edge, the orchestration of operations between Edge and Cloud requires an automated support, where context awareness plays a key role in assisting the network in deciding when and whereto store data and to perform computation. This work is focused on the application of context awareness to support a smoother operation of the Edge to Cloud operation, aiming at lowering latency, in particular when real time or close to real time data exchange is present.


Slightly Improved Upper Bound on the Integrality Ratio for the s t Path TSP. In this paper we investigate the integrality ratio of the standard LP relaxation for the metric s t Path TSP. We make a near optimal choice for an auxiliary function used in the analysis of Traub and Vygen which leads to an improved upper bound on the integrality ratio of 1.5273.


On the structure theory of cubespace fibrations. We study fibrations in the category of cubespaces nilspaces. We show that a fibration of finite degree f colon X rightarrow Y between compact ergodic gluing cubespaces (in particular nilspaces) factors as a (possibly countable) tower of compact abelian Lie group principal fiber bundles over Y . If the structure groups of f are connected then the fibers are (uniformly) isomorphic (in a strong sense) to an inverse limit of nilmanifolds. In addition we give conditions under which the fibers of f are isomorphic as subcubespaces. We introduce regionally proximal equivalence relations relative to factor maps between minimal topological dynamical systems for an arbitrary acting group. We prove that any factor map between minimal distal systems is a fibration and conclude that if such a map is of finite degree then it factors as a (possibly countable) tower of principal abelian Lie compact group extensions, thus achieving a refinement of both the Furstenberg's and the Bronstein Ellis structure theorems in this setting.


On the difference between entropic cost and the optimal transport cost. Consider the Monge Kantorovich problem of transporting densities rho 0 to rho 1 on mathbb R d with a strictly convex cost function. A popular relaxation of the problem is the one parameter family called the entropic cost problem. The entropic cost K h , h 0 , is significantly faster to compute and h K h is known to converge to the optimal transport cost as h goes to zero. We are interested the rate of convergence. We show that the difference between K h and 1 h times the optimal cost of transport has a pointwise limit when transporting a compactly supported density to another that satisfies a few other technical restrictions. This limit is the relative entropy of rho 1 with respect to a Riemannian volume measure on mathbb R d that measures the local sensitivity of the transport map. For the quadratic Wasserstein transport, this relative entropy is exactly one half of the difference of entropies of rho 1 and rho 0 . In that case we complement the results of Adams et al., Duong et al, and Erbar et al. who all use gamma convergence. More surprisingly, we demonstrate that this difference of two entropies (plus the cost) is also the limit for the Dirichlet transport introduced recently by Pal and Wong. The latter can be thought of as a multiplicative analog of the Wasserstein transport and corresponds to a non local operator. It hints at an underlying gradient flow of entropy, in the sense of Jordan Kinderlehrer Otto, even when the cost function is not a metric. The proofs are based on Gaussian approximations to Schr "odinger bridges as h approaches zero.


Enrichment of the nonconforming virtual element method with singular functions. We construct a nonconforming virtual element method (ncVEM) based on approximation spaces that are enriched with special singular functions. This enriched ncVEM is tailored for the approximation of solutions to elliptic problems, which have singularities due to the geometry of the domain. Differently from the traditional extended Galerkin method approach, based on the enrichment of local spaces with singular functions, no partition of unity is employed. Rather, the design of the method hinges upon the special structure of the nonconforming virtual element spaces. We discuss the theoretical analysis of the method and support it with several numerical experiments. We also present an orthonormalization procedure drastically trimming the ill conditioning of the final system.


Weakly Supervised Semantic Segmentation Using Constrained Dominant Sets. The availability of large scale data sets is an essential pre requisite for deep learning based semantic segmentation schemes. Since obtaining pixel level labels is extremely expensive, supervising deep semantic segmentation networks using low cost weak annotations has been an attractive research problem in recent years. In this work, we explore the potential of Constrained Dominant Sets (CDS) for generating multi labeled full mask predictions to train a fully convolutional network (FCN) for semantic segmentation. Our experimental results show that using CDS's yields higher quality mask predictions compared to methods that have been adopted in the literature for the same purpose.


Pointers in Recursion Exploring the Tropics. We translate the usual class of partial primitive recursive functions to a pointer recursion framework, accessing actual input values via a pointer reading unit cost function. These pointer recursive functions classes are proven equivalent to the usual partial primitive recursive functions. Complexity wise, this framework captures in a streamlined way most of the relevant sub polynomial classes. Pointer recursion with the safe normal tiering discipline of Bellantoni and Cook corresponds to polylogtime computation. We introduce a new, non size increasing tiering discipline, called tropical tiering. Tropical tiering and pointer recursion, used with some of the most common recursion schemes, capture the classes logspace, logspace polylogtime, ptime, and NC. Finally, in a fashion reminiscent of the safe recursive functions, tropical tiering is expressed directly in the syntax of the function algebras, yielding the tropical recursive function algebras.


Retrieving Sequential Information for Non Autoregressive Neural Machine Translation. Non Autoregressive Transformer (NAT) aims to accelerate the Transformer model through discarding the autoregressive mechanism and generating target words independently, which fails to exploit the target sequential information. Over translation and under translation errors often occur for the above reason, especially in the long sentence translation scenario. In this paper, we propose two approaches to retrieve the target sequential information for NAT to enhance its translation ability while preserving the fast decoding property. Firstly, we propose a sequence level training method based on a novel reinforcement algorithm for NAT (Reinforce NAT) to reduce the variance and stabilize the training procedure. Secondly, we propose an innovative Transformer decoder named FS decoder to fuse the target sequential information into the top layer of the decoder. Experimental results on three translation tasks show that the Reinforce NAT surpasses the baseline NAT system by a significant margin on BLEU without decelerating the decoding speed and the FS decoder achieves comparable translation performance to the autoregressive Transformer with considerable speedup.


Toward Terabits per second Communications Low Complexity Parallel Decoding of G N Coset Codes. Recently, a parallel decoding framework of G N coset codes was proposed. High throughput is achieved by decoding the independent component polar codes in parallel. Various algorithms can be employed to decode these component codes, enabling a flexible throughput performance tradeoff. In this work, we adopt SC as the component decoders to achieve the highest throughput end of the tradeoff. The benefits over soft output component decoders are reduced complexity and simpler (binary) interconnections among component decoders. To reduce performance degradation, we integrate an error detector and a log likelihood ratio (LLR) generator into each component decoder. The LLR generator, specifically the damping factors therein, is designed by a genetic algorithm. This low complexity design can achieve an area efficiency of 533Gbps mm 2 under 7nm technology.


Almost normally torsionfree ideals. We describe all connected graphs whose edge ideals are almost normally torsionfree. We also prove that the facet ideal of a special odd cycle is almost normally torsionfree. Finally, we determine the t spread principal Borel ideals generated in degree 3 which are almost normally torsionfree.


Design of a Solver for Multi Agent Epistemic Planning. As the interest in Artificial Intelligence continues to grow it is becoming more and more important to investigate formalization and tools that allow us to exploit logic to reason about the world. In particular, given the increasing number of multi agents systems that could benefit from techniques of automated reasoning, exploring new ways to define not only the world's status but also the agents' information is constantly growing in importance. This type of reasoning, i.e., about agents' perception of the world and also about agents' knowledge of her and others' knowledge, is referred to as epistemic reasoning. In our work we will try to formalize this concept, expressed through epistemic logic, for dynamic domains. In particular we will attempt to define a new action based language for multi agent epistemic planning and to implement an epistemic planner based on it. This solver should provide a tool flexible enough to be able to reason on different domains, e.g., economy, security, justice and politics, where reasoning about others' beliefs could lead to winning strategies or help in changing a group of agents' view of the world.


Generalized Expected Discounted Penalty Function at General Drawdown for L ' e vy Risk Processes. This paper considers an insurance surplus process modeled by a spectrally negative L ' e vy process. Instead of the time of ruin in the traditional setting, we apply the time of drawdown as the risk indicator in this paper. We study the joint distribution of the time of drawdown, the running maximum at drawdown, the last minimum before drawdown, the surplus before drawdown and the surplus at drawdown (may not be deficit in this case), which generalizes the known results on the classical expected discounted penalty function in Gerber and Shiu (1998). The results have semi explicit expressions in terms of the q scale functions and the L ' e vy measure associated with the L ' e vy process. As applications, the obtained result is applied to recover results in the literature and to obtain new results for the Gerber Shiu function at ruin for risk processes embedded with a loss carry forward taxation system or a barrier dividend strategy. Moreover, numerical examples are provided to illustrate the results.


Risk, Trust, and Bias Causal Regulators of Biometric Enabled Decision Support. Biometrics and biometric enabled decision support systems (DSS) have become a mandatory part of complex dynamic systems such as security checkpoints, personal health monitoring systems, autonomous robots, and epidemiological surveillance. Risk, trust, and bias (R T B) are emerging measures of performance of such systems. The existing studies on the R T B impact on system performance mostly ignore the complementary nature of R T B and their causal relationships, for instance, risk of trust, risk of bias, and risk of trust over biases. This paper offers a complete taxonomy of the R T B causal performance regulators for the biometric enabled DSS. The proposed novel taxonomy links the R T B assessment to the causal inference mechanism for reasoning in decision making. Practical details of the R T B assessment in the DSS are demonstrated using the experiments of assessing the trust in synthetic biometric and the risk of bias in face biometrics. The paper also outlines the emerging applications of the proposed approach beyond biometrics, including decision support for epidemiological surveillance such as for COVID 19 pandemics.


Fundamental Limits of Wireless Caching under Uneven Capacity Channels. This work identifies the fundamental limits of cache aided coded multicasting in the presence of the well known worst user' bottleneck. This stems from the presence of receiving users with uneven channel capacities, which often forces the rate of transmission of each multicasting message to be reduced to that of the slowest user. This bottleneck, which can be detrimental in general wireless broadcast settings, motivates the analysis of coded caching over a standard Single Input Single Output (SISO) Broadcast Channel (BC) with K cache aided receivers, each with a generally different channel capacity. For this setting, we design a communication algorithm that is based on superposition coding that capitalizes on the realization that the user with the worst channel may not be the real bottleneck of communication. We then proceed to provide a converse that shows the algorithm to be near optimal, identifying the fundamental limits of this setting within a multiplicative factor of 4. Interestingly, the result reveals that, even if several users are experiencing channels with reduced capacity, the system can achieve the same optimal delivery time that would be achievable if all users enjoyed maximal capacity.


From non Defectivity to Identifiability. A projective variety X subset mathbb P N is h identifiable if the generic element in its h secant variety uniquely determines h points on X . In this paper we propose an entirely new approach to study identifiability, connecting it to the notion of secant defect. In this way we are able to improve all known bounds on identifiability. In particular we give optimal bounds for some Segre and Segre Veronese varieties and provide the first identifiability statements for Grassmann varieties.


A refinement of the Ozsv 'ath Szab 'o large integer surgery formula and knot concordance. We compute the knot Floer filtration induced by a cable of the meridian of a knot in the manifold obtained by large integer surgery along the knot. We give a formula in terms of the original knot Floer complex of the knot in the three sphere. As an application, we show that a knot concordance invariant of Hom can equivalently be defined in terms of filtered maps on the Heegaard Floer homology groups induced by the two handle attachment cobordism of surgery along a knot.


Stochastic gradient descent for linear least squares problems with partially observed data. We propose a novel stochastic gradient descent method for solving linear least squares problems with partially observed data. Our method uses submatrices indexed by a randomly selected pair of row and column index sets to update the iterate at each step. Theoretical convergence guarantees in the mean square sense are provided. Numerical experiments are reported to demonstrate the theoretical findings.


A criterion for the existence of periodic points based on the eigenvalues of maps induced in cohomology. We present a criterion for the existence of periodic points based on the eigenvalues of maps induced in cohomology for spaces with rational cohomology isomorphic to a tensor product of a graded exterior algebra with generators in odd dimensions and a graded algebra with all elements of even degree. We give a number of natural examples of such spaces and provide some non trivial ones. We also give a counterexample to a claim in cite duan given there without proof.


Comparison of quiver varieties, loop Grassmannians and nilpotent cones in type A. In type A we find equivalences of geometries arising in three settings Nakajima's ( framed'') quiver varieties, conjugacy classes of matrices and loop Grassmannians. These are now all given by explicit formulas. As an application we provide a geometric version of symmetric and skew (GL(m), GL(n)) dualities.


Black boxing the user internet protocol over xylophone players (IPoXP). We introduce IP over Xylophone Players (IPoXP), a novel Internet protocol between two computers using xylophone based Arduino interfaces. In our implementation, human operators are situated within the lowest layer of the network, transmitting data between computers by striking designated keys. We discuss how IPoXP inverts the traditional mode of human computer interaction, with a computer using the human as an interface to communicate with another computer.


Supplementary Variable Method for Developing Structure Preserving Numerical Approximations to Thermodynamically Consistent Partial Differential Equations. We present a new temporal discretization paradigm for developing energy production rate preserving numerical approximations to thermodynamically consistent partial differential equation systems, called the supplementary variable method. The central idea behind it is to introduce a supplementary variable to the thermodynamically consistent model to make the over determined equation system, consisting of the thermodynamically consistent PDE system, the energy definition and the energy dissipation equation, structurally stable. The supplementary variable allows one to retain the consistency between the energy dissipation equation and the PDE system after the temporal discretization. We illustrate the method using a dissipative gradient flow model. Among virtually infinite many possibilities, we present two ways to add the supplementary variable in the gradient flow model to develop energy dissipation rate preserving algorithms. Spatial discretizations are carried out using the pseudo spectral method. We then compare the two new schemes with the energy stable SAV scheme and the fully implicit Crank Nicolson scheme. The results favor the new schemes in the overall performance. This new numerical paradigm can be applied to any thermodynamically consistent models.


Quasimodularity of the k th Residual Cranks. We establish quasimodularity for a family of residual crank generating functions defined on overpartitions. We also show that the second moments of these k th residual cranks admit a combinatoric interpretation as weighted overpartition counts.


UDS DFKI Submission to the WMT2019 Similar Language Translation Shared Task. In this paper we present the UDS DFKI system submitted to the Similar Language Translation shared task at WMT 2019. The first edition of this shared task featured data from three pairs of similar languages Czech and Polish, Hindi and Nepali, and Portuguese and Spanish. Participants could choose to participate in any of these three tracks and submit system outputs in any translation direction. We report the results obtained by our system in translating from Czech to Polish and comment on the impact of out of domain test data in the performance of our system. UDS DFKI achieved competitive performance ranking second among ten teams in Czech to Polish translation.


On Agemi type structural conditions for a system of semilinear wave equations. We consider a two component system of cubic semilinear wave equations in two space dimensions satisfying the Agemi type structural condition (Ag) but violating (Ag 0 ) and (Ag ). For this system, we show that small amplitude solutions are asymptotically free as t to infty .


Dialogue Relation Extraction with Document level Heterogeneous Graph Attention Networks. Dialogue relation extraction (DRE) aims to detect the relation between two entities mentioned in a multi party dialogue. It plays an important role in constructing knowledge graphs from conversational data increasingly abundant on the internet and facilitating intelligent dialogue system development. The prior methods of DRE do not meaningfully leverage speaker information they just prepend the utterances with the respective speaker names. Thus, they fail to model the crucial inter speaker relations that may give additional context to relevant argument entities through pronouns and triggers. We, however, present a graph attention network based method for DRE where a graph, that contains meaningfully connected speaker, entity, entity type, and utterance nodes, is constructed. This graph is fed to a graph attention network for context propagation among relevant nodes, which effectively captures the dialogue context. We empirically show that this graph based approach quite effectively captures the relations between different entity pairs in a dialogue as it outperforms the state of the art approaches by a significant margin on the benchmark dataset DialogRE. Our code is released at https github.com declare lab dialog HGAT


Iris Presentation Attack Detection Where Are We Now?. As the popularity of iris recognition systems increases, the importance of effective security measures against presentation attacks becomes paramount. This work presents an overview of the most important advances in the area of iris presentation attack detection published in recent two years. Newly released, publicly available datasets for development and evaluation of iris presentation attack detection are discussed. Recent literature can be seen to be broken into three categories traditional "hand crafted" feature extraction and classification, deep learning based solutions, and hybrid approaches fusing both methodologies. Conclusions of modern approaches underscore the difficulty of this task. Finally, commentary on possible directions for future research is provided.


Unsupervised Monocular Depth Prediction for Indoor Continuous Video Streams. This paper studies unsupervised monocular depth prediction problem. Most of existing unsupervised depth prediction algorithms are developed for outdoor scenarios, while the depth prediction work in the indoor environment is still very scarce to our knowledge. Therefore, this work focuses on narrowing the gap by firstly evaluating existing approaches in the indoor environments and then improving the state of the art design of architecture. Unlike typical outdoor training dataset, such as KITTI with motion constraints, data for indoor environment contains more arbitrary camera movement and short baseline between two consecutive images, which deteriorates the network training for the pose estimation. To address this issue, we propose two methods Firstly, we propose a novel reconstruction loss function to constraint pose estimation, resulting in accuracy improvement of the predicted disparity map secondly, we use an ensemble learning with a flipping strategy along with a median filter, directly taking operation on the output disparity map. We evaluate our approaches on the TUM RGB D and self collected datasets. The results have shown that both approaches outperform the previous state of the art unsupervised learning approaches.


Sharon Shared Online Event Sequence Aggregation. Streaming systems evaluate massive workloads of event sequence aggregation queries. State of the art approaches suffer from long delays caused by not sharing intermediate results of similar queries and by constructing event sequences prior to their aggregation. To overcome these limitations, our Shared Online Event Sequence Aggregation (Sharon) approach shares intermediate aggregates among multiple queries while avoiding the expensive construction of event sequences. Our Sharon optimizer faces two challenges. One, a sharing decision is not always beneficial. Two, a sharing decision may exclude other sharing opportunities. To guide our Sharon optimizer, we compactly encode sharing candidates, their benefits, and conflicts among candidates into the Sharon graph. Based on the graph, we map our problem of finding an optimal sharing plan to the Maximum Weight Independent Set (MWIS) problem. We then use the guaranteed weight of a greedy algorithm for the MWIS problem to prune the search of our sharing plan finder without sacrificing its optimality. The Sharon optimizer is shown to produce sharing plans that achieve up to an 18 fold speed up compared to state of the art approaches.


Subtractive Perceptrons for Learning Images A Preliminary Report. In recent years, artificial neural networks have achieved tremendous success for many vision based tasks. However, this success remains within the paradigm of emph weak AI where networks, among others, are specialized for just one given task. The path toward emph strong AI , or Artificial General Intelligence, remains rather obscure. One factor, however, is clear, namely that the feed forward structure of current networks is not a realistic abstraction of the human brain. In this preliminary work, some ideas are proposed to define a textit subtractive Perceptron (s Perceptron), a graph based neural network that delivers a more compact topology to learn one specific task. In this preliminary study, we test the s Perceptron with the MNIST dataset, a commonly used image archive for digit recognition. The proposed network achieves excellent results compared to the benchmark networks that rely on more complex topologies.


On mathcal I( q) and mathcal I( leq q) convergence of arithmetic functions. Let mathbb N be the set of positive integers, and denote by lambda(A) inf t 0 sum a in A a t infty the convergence exponent of A subset mathbb N . For 0 q le 1 , 0 le q le 1 , respectively, the admissible ideals mathcal I( q) , mathcal I( leq q) of all subsets A subset mathbb N with lambda(A) q , lambda(A) le q , respectively, satisfy mathcal I( q) subsetneq mathcal I c (q) subsetneq mathcal I( leq q) , where mathcal I c (q) A subset mathbb N sum a in A a q infty . In this note we sharpen the results of Bal 'az, Gogola and Visnyai from 2 , and of others papers, concerning characterizations of mathcal I c (q) convergence of various arithmetic functions in terms of q . This is achieved by utilizing mathcal I( q) and mathcal I( leq q) convergence, for which new methods and criteria are developed.


Finding new routes for integrating Multi Agent Systems using Apache Camel. In Multi Agent Systems (MAS) there are two main models of interaction among agents, and between agents and the environment. Although there are studies considering these models, there is no practical tool to afford the interaction with external entities with both models. This paper presents a proposal for such a tool based on the Apache Camel framework by designing two new components, namely camel jason and camel artifact. By means of these components, an external entity is modelled according to its nature, i.e., whether it is autonomous or non autonomous, interacting with the MAS respectively as an agent or an artifact. It models coherently external entities whereas Camel provides interoperability with several communication protocols.


Tantala Kerr nonlinear integrated photonics. Integrated photonics plays a central role in modern science and technology, enabling experiments from nonlinear science to quantum information, ultraprecise measurements and sensing, and advanced applications like data communication and signal processing. Optical materials with favorable properties are essential for nanofabrication of integrated photonics devices. Here we describe a material for integrated nonlinear photonics, tantalum pentoxide (Ta 2 O 5 , hereafter tantala), which offers low intrinsic material stress, low optical loss, and efficient access to Kerr nonlinear processes. We utilize 800 nm thick tantala films deposited via ion beam sputtering on oxidized silicon wafers. The tantala films contain a low residual tensile stress of 38 MPa, and they offer a Kerr index n 2 6.2(23) times10 19 m 2 W, which is approximately a factor of three higher than silicon nitride. We fabricate integrated nonlinear resonators and waveguides without the cracking challenges that are prevalent in stoichiometric silicon nitride. The tantala resonators feature an optical quality factor up to 3.8 times10 6 , which enables us to generate ultrabroad bandwidth Kerr soliton frequency combs with low threshold power. Moreover, tantala waveguides enable supercontinuum generation across the near infrared from low energy, ultrafast seed pulses. Our work introduces a versatile material platform for integrated, low loss nanophotonics that can be broadly applied and enable heterogeneous integration.


Uniform Welfare Guarantees Under Identical Subadditive Valuations. We study the problem of allocating indivisible goods among agents that have an identical subadditive valuation over the goods. The extent of fairness and efficiency of allocations is measured by the generalized means of the values that the allocations generate among the agents. Parameterized by an exponent term p , generalized mean welfares encompass multiple well studied objectives, such as social welfare, Nash social welfare, and egalitarian welfare. We establish that, under identical subadditive valuations and in the demand oracle model, one can efficiently find a single allocation that approximates the optimal generalized mean welfare to within a factor of 40 uniformly for all p in ( infty, 1 . Hence, by way of a constant factor approximation algorithm, we obtain novel results for maximizing Nash social welfare and egalitarian welfare for identical subadditive valuations.


Machine Learning and Deep Learning methods for predictive modelling from Raman spectra in bioprocessing. In chemical processing and bioprocessing, conventional online sensors are limited to measure only basic process variables like pressure and temperature, pH, dissolved O and CO 2 and viable cell density (VCD). The concentration of other chemical species is more difficult to measure, as it usually requires an at line or off line approach. Such approaches are invasive and slow compared to on line sensing. It is known that different molecules can be distinguished by their interaction with monochromatic light, producing different profiles for the resulting Raman spectrum, depending on the concentration. Given the availability of reference measurements for the target variable, regression methods can be used to model the relationship between the profile of the Raman spectra and the concentration of the analyte. This work focused on pretreatment methods of Raman spectra for the facilitation of the regression task using Machine Learning and Deep Learning methods, as well as the development of new regression models based on these methods. In the majority of cases, this allowed to outperform conventional Raman models in terms of prediction error and prediction robustness.


Small angle limits of Hamilton's footballs. Compact Ricci solitons on surfaces have at most two cone points, and are known as Hamilton's footballs. In this note we completely describe the degenerations of these footballs as one or both of the cone angles approaches zero. In particular, we show that Hamilton's famous non compact cigar soliton is the Gromov Hausdorff limit of Hamilton's compact conical teardrop solitons.


The Exponential of the S 1 Trace of the Free Field and Verblunsky Coefficients. An identity of Szego, and a volume calculation, heuristically suggest a simple expression for the distribution of Verblunsky coefficients with respect to the (normalized) exponential of the S 1 trace of the Gaussian free field. This heuristic expression is not quite correct. A proof of the correct formula has been found by Chhaibi and Najnudel. Their proof uses random matrix theory and overcomes many difficult technical issues. In addition to presenting the Szego perspective, we show that the Chhaibi and Najnudel theorem implies a family of combinatorial identities (for moments of measures) which are of intrinsic interest.


Pentest on an Internet Mobile App A Case Study using Tramonto. Mobile applications are used to handle different types of data. Commonly, there is a set of personal identifiable information present in the data stored, shared and used by these applications. From that, attackers can try to exploit the mobile application in order to obtain or to cause private data leakage. Therefore, performing security assessments is an important practice to find vulnerabilities in the applications and systems before the application is deployed, or even during their use. Regarding security assessments, Penetration Test (Pentest) is one of the security test types that can be used to detect vulnerabilities through simulated attacks. Additionally, Pentest can be performed using different methodologies and best practices, through several frameworks to organize the test execution, execute tools, provide estimations, provide reports and document a Pentest. One such framework is Tramonto, which aims to assist a cybersecurity expert during the Pentest execution by providing organization, standardization and flexibility to the whole Pentest process. This paper presents a Pentest case study applied to a Brazilian university Mobile App using the Tramonto framework. The main goal of this case study is to present how Tramonto can be applied during a Pentest execution, assisting cybersecurity experts in the tasks included in the Pentest process. Our results show details on how to perform a Pentest using Tramonto and the found vulnerabilities in the Mobile App. Besides that, there is a discussion about the main contributions obtained from our results, and we were able to verify that Tramonto managed, organized and optimized the whole Pentest process.


Experiments in Autonomous Driving Through Imitation Learning. This report demonstrates several methods used to make a self driving vehicle using a supervised learning algorithm and a forward facing RGBD camera. The project originally involved research in creating an adversarial attack on the vehicle's model, but due to difficulties with the initial training of the car, the plans were discarded in favor of completing the imitation learning portion of the project. Many approaches were explored, but due to challenges introduced by an unbalanced data set, the approaches had limited effectiveness.


Non Canonical GC Base Pairs and Mechanochemical Cleavage of DNA. Properties of non canonical GC Base Pairs and their relation with mechanochemical cleavage of DNA are analyzed. A hypothesis of the involvement of the Transient GC Wobble Base Pairs in the mechanisms of the mechanochemical cleavage of DNA and epigenetic mechanisms with participation of 5 methylcytosine is proposed. The hypothesis explains the increase in the frequency of the breaks of the sugar phosphate backbone of DNA after cytosines, asymmetric character of these breaks, and an increase in the frequency of breaks in CpG after cytosine methylation.


PPaaS Privacy Preservation as a Service. Personally identifiable information (PII) can find its way into cyberspace through various channels, and many potential sources can leak such information. Data sharing (e.g. cross agency data sharing) for machine learning and analytics is one of the important components in data science. However, due to privacy concerns, data should be enforced with strong privacy guarantees before sharing. Different privacy preserving approaches were developed for privacy preserving data sharing however, identifying the best privacy preservation approach for the privacy preservation of a certain dataset is still a challenge. Different parameters can influence the efficacy of the process, such as the characteristics of the input dataset, the strength of the privacy preservation approach, and the expected level of utility of the resulting dataset (on the corresponding data mining application such as classification). This paper presents a framework named underline P rivacy underline P reservation underline a s underline a underline S ervice (PPaaS) to reduce this complexity. The proposed method employs selective privacy preservation via data perturbation and looks at different dynamics that can influence the quality of the privacy preservation of a dataset. PPaaS includes pools of data perturbation methods, and for each application and the input dataset, PPaaS selects the most suitable data perturbation approach after rigorous evaluation. It enhances the usability of privacy preserving methods within its pool it is a generic platform that can be used to sanitize big data in a granular, application specific manner by employing a suitable combination of diverse privacy preserving algorithms to provide a proper balance between privacy and utility.


Algebraic criteria for stable diffeomorphism of spin 4 manifolds. We study closed, connected, spin 4 manifolds up to stabilisation by connected sums with copies of S 2 times S 2 . For a fixed fundamental group, there are primary, secondary and tertiary obstructions, which together with the signature lead to a complete stable classification. The primary obstruction exactly detects mathbb CP 2 stable diffeomorphism and was previously related to algebraic invariants by Kreck and the authors. In this article we formulate conjectural relationships of the secondary and tertiary obstructions with algebraic invariants the secondary obstruction should be determined by the (stable) equivariant intersection form and the tertiary obstruction via a tau invariant recording intersection data between 2 spheres, with trivial algebraic self intersection, and their Whitney discs. We prove our conjectures for the following classes of fundamental groups groups of cohomological dimension at most 3, right angled Artin groups, abelian groups, and finite groups with quaternion or abelian 2 Sylow subgroups. We apply our theory to give a complete algebraic stable classification of spin 4 manifolds with fundamental group mathbb Z times mathbb Z 2 .


SEDRo A Simulated Environment for Developmental Robotics. Even with impressive advances in application specific models, we still lack knowledge about how to build a model that can learn in a human like way and do multiple tasks. To learn in a human like way, we need to provide a diverse experience that is comparable to humans. In this paper, we introduce our ongoing effort to build a simulated environment for developmental robotics (SEDRo). SEDRo provides diverse human experiences ranging from those of a fetus to a 12th month old. A series of simulated tests based on developmental psychology will be used to evaluate the progress of a learning model. We anticipate SEDRo to lower the cost of entry and facilitate research in the developmental robotics community.


Node Overlap Removal Algorithms A Comparative Study. Many algorithms have been designed to remove node overlapping, and many quality criteria and associated metrics have been proposed to evaluate those algorithms. Unfortunately, a complete comparison of the algorithms based on some metrics that evaluate the quality has never been provided and it is thus difficult for a visualization designer to select the algorithm that best suits his needs. In this paper, we review 21 metrics available in the literature, classify them according to the quality criteria they try to capture, and select a representative one for each class. Based on the selected metrics, we compare 8 node overlap removal algorithms. Our experiment involves 854 synthetic and real world graphs.


An example of rotationally symmetric Q n 1 translators and a non existence theorem in mathbb R n 1 . The main result in this paper is a non existence Theorem of entire Q n 1 translators in mathbb R n 1 . In addition, an example of non entire complete Q n 1 translator has been found and a Tangential Principle for Q k translators in mathbb R n 1 .


On small scale and large scale intermittency of Lagrangian statistics in canopy flow. The interaction of fluids with surface mounted obstacles in canopy flows leads to strong turbulence that dominates dispersion and mixing in the neutrally stable atmospheric surface layer. This work focuses on intermittency in the Lagrangian velocity statistics in a canopy flow, which is observed in two distinct forms. The first, small scale intermittency, is expressed by non Gaussian and not self similar statistics of the velocity increments. The analysis shows an agreement in comparison with previous results from homogeneous isotropic turbulence (HIT) using the multifractal model, extended self similarity, and acceleration autocorrelations. These observations suggest that the picture of small scale Lagrangian intermittency in canopy flows is similar to that in HIT, and therefore, they extend the idea of universal Lagrangian intermittency to certain inhomogeneous and anisotropic flows. Second, it is observed that the RMS of energy increments along Lagrangian trajectories depend on the direction of the trajectories' time averaged turbulent velocity. Subsequent analysis suggests that the flow is attenuated by the canopy drag while leaving the structure function's scaling unchanged. This observation implies the existence of large scale intermittency in Lagrangian statistics. Thus, this work presents a first empirical evidence of intermittent Lagrangian velocity statistics in a canopy flow that exists in two distinct senses and occurs due to different mechanisms.


Gabor Duality Theory for Morita Equivalent C algebras. The duality principle for Gabor frames is one of the pillars of Gabor analysis. We establish a far reaching generalization to Morita equivalent C algebras where the equivalence bimodule is a finitely generated projective Hilbert C module. These Hilbert C modules are equipped with some extra structure and are called Gabor bimodules. We formulate a duality principle for standard module frames for Gabor bimodules which reduces to the well known Gabor duality principle for twisted group C algebras of a lattice in phase space. We lift all these results to the matrix algebra level and in the description of the module frames associated to a matrix Gabor bimodule we introduce (n,d) matrix frames, which generalize superframes and multi window frames. Density theorems for (n,d) matrix frames are established, which extend the ones for multi window and super Gabor frames. Our approach is based on the localization of a Hilbert C module with respect to a trace.


Multi Person tracking by multi scale detection in Basketball scenarios. Tracking data is a powerful tool for basketball teams in order to extract advanced semantic information and statistics that might lead to a performance boost. However, multi person tracking is a challenging task to solve in single camera video sequences, given the frequent occlusions and cluttering that occur in a restricted scenario. In this paper, a novel multi scale detection method is presented, which is later used to extract geometric and content features, resulting in a multi person video tracking system. Having built a dataset from scratch together with its ground truth (more than 10k bounding boxes), standard metrics are evaluated, obtaining notable results both in terms of detection (F1 score) and tracking (MOTA). The presented system could be used as a source of data gathering in order to extract useful statistics and semantic analyses a posteriori.


On a model of evolution of subspecies. Ben Ari and Schinazi (2016) introduced a stochastic model to study virus like evolving population with high mutation rate'. This model is a birth and death model with an individual at birth being either a mutant with a random fitness parameter in 0,1 or having one of the existing fitness parameters with uniform probability whereas a death event removes the entire population of the least fit site. We change this to incorporate the notion of survival of the fittest', by requiring that a non mutant individual, at birth, has a fitness according to a preferential attachment mechanism, i.e., it has a fitness f with a probability proportional to the size of the population of fitness f . Also death just removes one individual at the least fit site. This preferential attachment rule leads to a power law behaviour in the asymptotics, unlike the exponential behaviour obtained by Ben Ari and Schinazi (2016).


Box Suite Recommendation. An algorithm for recommending a suite of boxes for shipping a retailer's online customer orders is presented.


Testing Product Distributions A Closer Look. We study the problems of identity and closeness testing of n dimensional product distributions. Prior works by Canonne, Diakonikolas, Kane and Stewart (COLT 2017) and Daskalakis and Pan (COLT 2017) have established tight sample complexity bounds for non tolerant testing over a binary alphabet given two product distributions P and Q over a binary alphabet, distinguish between the cases P Q and d mathrm TV (P, Q) epsilon . We build on this prior work to give a more comprehensive map of the complexity of testing of product distributions by investigating tolerant testing with respect to several natural distance measures and over an arbitrary alphabet. Our study gives a fine grained understanding of how the sample complexity of tolerant testing varies with the distance measures for product distributions. In addition, we also extend one of our upper bounds on product distributions to bounded degree Bayes nets.


Representations of the fixed point subalgebra of the vertex algebra associated to a non degenerate even lattice by an automorphism of order 2 . Let V L be the vertex algebra associated to a non degenerate even lattice L , theta the automorphism of V L induced from the 1 isometry of L , and V L the fixed point subalgebra of V L under the action of theta . We show that every weak V L module is completely reducible.


Primal superlinear convergence of SQP methods in piecewise linear quadratic composite optimization. This paper mainly concerns with the primal superlinear convergence of the quasi Newton sequential quadratic programming (SQP) method for piecewise linear quadratic composite optimization problems. We show that the latter primal superlinear convergence can be justified under the noncriticality of Lagrange multipliers and a version of the Dennis More condition. Furthermore, we show that if we replace the noncriticality condition with the second order sufficient condition, this primal superlinear convergence is equivalent with an appropriate version of the Dennis More condition. We also recover Bonnans' result in 1 for the primal dual superlinear of the basic SQP method for this class of composite problems under the second order sufficient condition and the uniqueness of Lagrange multipliers. To achieve these goals, we first obtain an extension of the reduction lemma for convex Piecewise linear quadratic functions and then provide a comprehensive analysis of the noncriticality of Lagrange multipliers for composite problems. We also establish certain primal estimates for KKT systems of composite problems, which play a significant role in our local convergence analysis of the quasi Newton SQP method.


A "right" path to cyclic polygons. It is well known that Heron's theorem provides an explicit formula for the area of a triangle, as a symmetric function of the lengths of its sides. It has been extended by Brahmagupta to quadrilaterals inscribed in a circle (cyclic quadrilaterals). A natural problem is trying to further generalize the result to cyclic polygons with a larger number of edges, which, surprisingly, has revealed to be far from simple. In this paper we investigate such a problem by following a new and elementary approach. We start from the simple observation that the incircle of a right triangle touches its hypothenuse in a point that splits it into two segments, the product of whose lengths equals the area of the triangle. From this curious fact we derive in a few lines an unusual proof of the Pythagoras' theorem, Heron's theorem for right triangles, Heron's theorem for general triangles, and Brahmagupta's theorem for cyclic quadrangles. This suggests that cutting the edges of a cyclic polygon by means of suitable points should be the "right" working method. Indeed, following this idea, we obtain an explicit formula for the area of any convex cyclic polygon, as a symmetric function of the segments split on its edges by the incircles of a triangulation. We also show that such a symmetry can be rediscovered in Heron's and Brahmagupta's results, which consequently represent special cases of the general provided formula.


AssembleNet Assembling Modality Representations via Attention Connections. We create a family of powerful video models which are able to (i) learn interactions between semantic object information and raw appearance and motion features, and (ii) deploy attention in order to better learn the importance of features at each convolutional block of the network. A new network component named peer attention is introduced, which dynamically learns the attention weights using another block or input modality. Even without pre training, our models outperform the previous work on standard public activity recognition datasets with continuous videos, establishing new state of the art. We also confirm that our findings of having neural connections from the object modality and the use of peer attention is generally applicable for different existing architectures, improving their performances. We name our model explicitly as AssembleNet . The code will be available at https sites.google.com corp view assemblenet


Reducing Anomaly Detection in Images to Detection in Noise. Anomaly detectors address the difficult problem of detecting automatically exceptions in an arbitrary background image. Detection methods have been proposed by the thousands because each problem requires a different background model. By analyzing the existing approaches, we show that the problem can be reduced to detecting anomalies in residual images (extracted from the target image) in which noise and anomalies prevail. Hence, the general and impossible background modeling problem is replaced by simpler noise modeling, and allows the calculation of rigorous thresholds based on the a contrario detection theory. Our approach is therefore unsupervised and works on arbitrary images.


Pump and Dumps in the Bitcoin Era Real Time Detection of Cryptocurrency Market Manipulations. In the last years, cryptocurrencies are increasingly popular. Even people who are not experts have started to invest in these securities and nowadays cryptocurrency exchanges process transactions for over 100 billion US dollars per month. However, many cryptocurrencies have low liquidity and therefore they are highly prone to market manipulation schemes. In this paper, we perform an in depth analysis of pump and dump schemes organized by communities over the Internet. We observe how these communities are organized and how they carry out the fraud. Then, we report on two case studies related to pump and dump groups. Lastly, we introduce an approach to detect the fraud in real time that outperforms the current state of the art, so to help investors stay out of the market when a pump and dump scheme is in action.


Generating the Twist Subgroup by Involutions. For a nonorientable surface, the twist subgroup is an index 2 subgroup of the mapping class group. It is generated by Dehn twists about two sided simple closed curves. In this paper, we study involution generators of the twist subgroup. We give generating sets of involutions with the smallest number of elements our methods allow.


Morphology Agnostic Visual Robotic Control. Existing approaches for visuomotor robotic control typically require characterizing the robot in advance by calibrating the camera or performing system identification. We propose MAVRIC, an approach that works with minimal prior knowledge of the robot's morphology, and requires only a camera view containing the robot and its environment and an unknown control interface. MAVRIC revolves around a mutual information based method for self recognition, which discovers visual "control points" on the robot body within a few seconds of exploratory interaction, and these control points in turn are then used for visual servoing. MAVRIC can control robots with imprecise actuation, no proprioceptive feedback, unknown morphologies including novel tools, unknown camera poses, and even unsteady handheld cameras. We demonstrate our method on visually guided 3D point reaching, trajectory following, and robot to robot imitation.


Pano Optimizing 360 deg Video Streaming with a Better Understanding of Quality Perception. Streaming 360 deg videos requires more bandwidth than non 360 deg videos. This is because current solutions assume that users perceive the quality of 360 deg videos in the same way they perceive the quality of non 360 deg videos. This means the bandwidth demand must be proportional to the size of the user's field of view. However, we found several qualitydetermining factors unique to 360 deg videos, which can help reduce the bandwidth demand. They include the moving speed of a user's viewpoint (center of the user's field of view), the recent change of video luminance, and the difference in depth of fields of visual objects around the viewpoint. This paper presents Pano, a 360 deg video streaming system that leverages the 360 deg video specific factors. We make three contributions. (1) We build a new quality model for 360 deg videos that captures the impact of the 360 deg video specific factors. (2) Pano proposes a variable sized tiling scheme in order to strike a balance between the perceived quality and video encoding efficiency. (3) Pano proposes a new qualityadaptation logic that maximizes 360 deg video user perceived quality and is readily deployable. Our evaluation (based on user study and trace analysis) shows that compared with state of the art techniques, Pano can save 41 46 bandwidth without any drop in the perceived quality, or it can raise the perceived quality (user rating) by 25 142 without using more bandwidth.


An End to End ML System for Personalized Conversational Voice Models in Walmart E Commerce. Searching for and making decisions about products is becoming increasingly easier in the e commerce space, thanks to the evolution of recommender systems. Personalization and recommender systems have gone hand in hand to help customers fulfill their shopping needs and improve their experiences in the process. With the growing adoption of conversational platforms for shopping, it has become important to build personalized models at scale to handle the large influx of data and perform inference in real time. In this work, we present an end to end machine learning system for personalized conversational voice commerce. We include components for implicit feedback to the model, model training, evaluation on update, and a real time inference engine. Our system personalizes voice shopping for Walmart Grocery customers and is currently available via Google Assistant, Siri and Google Home devices.


AGL a Scalable System for Industrial purpose Graph Machine Learning. Machine learning over graphs have been emerging as powerful learning tools for graph data. However, it is challenging for industrial communities to leverage the techniques, such as graph neural networks (GNNs), and solve real world problems at scale because of inherent data dependency in the graphs. As such, we cannot simply train a GNN with classic learning systems, for instance parameter server that assumes data parallel. Existing systems store the graph data in memory for fast accesses either in a single machine or graph stores from remote. The major drawbacks are in three fold. First, they cannot scale because of the limitations on the volume of the memory, or the bandwidth between graph stores and workers. Second, they require extra development of graph stores without well exploiting mature infrastructures such as MapReduce that guarantee good system properties. Third, they focus on training but ignore the optimization of inference over graphs, thus makes them an unintegrated system. In this paper, we design AGL, a scalable, fault tolerance and integrated system, with fully functional training and inference for GNNs. Our system design follows the message passing scheme underlying the computations of GNNs. We design to generate the k hop neighborhood, an information complete subgraph for each node, as well as do the inference simply by merging values from in edge neighbors and propagating values to out edge neighbors via MapReduce. In addition, the k hop neighborhood contains information complete subgraphs for each node, thus we simply do the training on parameter servers due to data independency. Our system AGL, implemented on mature infrastructures, can finish the training of a 2 layer graph attention network on a graph with billions of nodes and hundred billions of edges in 14 hours, and complete the inference in 1.2 hour.


Orthogonal ell 1 sets and extreme non Arens regularity of preduals of von Neumann algebras. We propose a new definition for a Banach algebra mathfrak A to be extremely non Arens regular, namely that the quotient mathfrak A ast mathscr WAP ( mathfrak A ) of mathfrak A ast with the space of its weakly almost periodic elements contains an isomorphic copy of mathfrak A ast. This definition is simpler and formally stronger than the original one introduced by Granirer in the nineties. We then identify sufficient conditions for the predual mathfrak V ast of a von Neumann algebra mathfrak V to be extremely non Arens regular in this new sense. These conditions are obtained with the help of orthogonal ell 1 sets of mathfrak V ast. We show that some of the main algebras in Harmonic Analysis satisfy these conditions. Among them,there is small bullet the weighted semigroup algebra of any weakly cancellative discrete semigroup, for any diagonally bounded weight, small bullet the weighted group algebra of any non discrete locally compact infinite group and for any weight, small bullet the weighted measure algebra of any locally compact infinite group, for any diagonally bounded weight, small bullet the Fourier algebra of any locally compact infinite group having its local weight greater or equal than its compact covering number, small bullet the Fourier algebra of any countable discrete group containing an infinite amenable subgroup.


Mutual Information Maximization on Disentangled Representations for Differential Morph Detection. In this paper, we present a novel differential morph detection framework, utilizing landmark and appearance disentanglement. In our framework, the face image is represented in the embedding domain using two disentangled but complementary representations. The network is trained by triplets of face images, in which the intermediate image inherits the landmarks from one image and the appearance from the other image. This initially trained network is further trained for each dataset using contrastive representations. We demonstrate that, by employing appearance and landmark disentanglement, the proposed framework can provide state of the art differential morph detection performance. This functionality is achieved by the using distances in landmark, appearance, and ID domains. The performance of the proposed framework is evaluated using three morph datasets generated with different methodologies.


A locally modified second order finite element method for interface problems. The locally modified finite element method, which is introduced in Frei, Richter SINUM 52(2014), p. 2315 2334 is a simple fitted finite element method that is able to resolve weak discontinuities in interface problems. The method is based on a fixed structured coarse mesh, which is then refined into sub elements to resolve an interior interface. In this work, we extend the locally modified finite element method to second order using an isoparametric approach in the interface elements. Thereby we need to take care that the resulting curved edges do not lead to degenerate sub elements. We prove optimal a priori error estimates in the L 2 norm and in a modified energy norm, as well as a reduced convergence order of cal O (h 3 2 ) in the standard H 1 norm. Finally, we present numerical examples to substantiate the theoretical findings.


Far field subwavelength resolution imaging by spatial spectrum sampling. Imaging below the diffraction limit is always a public interest because of the restricted resolution of conventional imaging systems. To beat the limit, evanescent harmonics decaying in space must participate in the imaging process. Here, we introduce the method of spatial spectrum sampling, a novel far field superresolution imaging method for microwave and terahertz regime. Strong dispersion and momentum conservation allow the spoof surface plasmon polaritons (SSP) structure to become a sensitive probe for spatial harmonics. This enables that the spatial information of the targets including both propagating and evanescent components, can be extracted by tuning and recording SSP in the far field. Then, the subwavelength resolution is constructed by the inversed Fourier transform of the sampled spatial spectrum. Using the modified subwavelength metallic grating as the spoof plasmonic structure, a far field resolution of 0.17 wavelength is numerically and experimentally verified, and two dimensional imaging ability is also fully discussed. The imaging ability and flexibility can be further optimizing the SSP structures. We are confident that our working mechanism will have great potentials in the superresolution imaging applications in the microwave and terahertz frequency range


Label Confusion Learning to Enhance Text Classification Models. Representing a true label as a one hot vector is a common practice in training text classification models. However, the one hot representation may not adequately reflect the relation between the instances and labels, as labels are often not completely independent and instances may relate to multiple labels in practice. The inadequate one hot representations tend to train the model to be over confident, which may result in arbitrary prediction and model overfitting, especially for confused datasets (datasets with very similar labels) or noisy datasets (datasets with labeling errors). While training models with label smoothing (LS) can ease this problem in some degree, it still fails to capture the realistic relation among labels. In this paper, we propose a novel Label Confusion Model (LCM) as an enhancement component to current popular text classification models. LCM can learn label confusion to capture semantic overlap among labels by calculating the similarity between instances and labels during training and generate a better label distribution to replace the original one hot label vector, thus improving the final classification performance. Extensive experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models. Further experiments also verify that LCM is especially helpful for confused or noisy datasets and superior to the label smoothing method.


Triangle colorings require at least seven colors. We show that if a coloring of the plane has the properties that any two points at distance one are colored differently and the plane is partitioned into uniformly colored triangles under certain conditions, then it requires at least seven colors. This is also true for a coloring using uniformly colored polygons if it has a point bordering at least four polygons.


Kvistur 2.0 a BiLSTM Compound Splitter for Icelandic. In this paper, we present a character based BiLSTM model for splitting Icelandic compound words, and show how varying amounts of training data affects the performance of the model. Compounding is highly productive in Icelandic, and new compounds are constantly being created. This results in a large number of out of vocabulary (OOV) words, negatively impacting the performance of many NLP tools. Our model is trained on a dataset of 2.9 million unique word forms and their constituent structures from the Database of Icelandic Morphology. The model learns how to split compound words into two parts and can be used to derive the constituent structure of any word form. Knowing the constituent structure of a word form makes it possible to generate the optimal split for a given task, e.g., a full split for subword tokenization, or, in the case of part of speech tagging, splitting an OOV word until the largest known morphological head is found. The model outperforms other previously published methods when evaluated on a corpus of manually split word forms. This method has been integrated into Kvistur, an Icelandic compound word analyzer.


Lieb Thirring constant on the sphere and on the torus. We prove on the 2D sphere and on the 2D torus the Lieb Thirring inequalities with improved constants for orthonormal families of scalar and vector functions.


The Fibonacci Quilt Game. Zeckendorf proved that every positive integer can be expressed as the sum of non consecutive Fibonacci numbers. This theorem inspired a beautiful game, the Zeckendorf Game. Two players begin with n 1 's and take turns applying rules inspired by the Fibonacci recurrence, F n 1 F n F n 1 , until a decomposition without consecutive terms is reached whoever makes the last move wins. We look at a game resulting from a generalization of the Fibonacci numbers, the Fibonacci Quilt sequence. These arise from the two dimensional geometric property of tiling the plane through the Fibonacci spiral. Beginning with 1 in the center, we place integers in the squares of the spiral such that each square contains the smallest positive integer that does not have a decomposition as the sum of previous terms that do not share a wall. This sequence eventually follows two recurrence relations, allowing us to construct a variation on the Zeckendorf Game, the Fibonacci Quilt Game. While some properties of the Fibonaccis are inherited by this sequence, the nature of its recurrence leads to others, such as Zeckendorf's theorem, no longer holding it is thus of interest to investigate the generalization of the game in this setting to see which behaviors persist. We prove, similar to the original game, that this game also always terminates in a legal decomposition, give a lower bound on game lengths, show that depending on strategies the length of the game can vary and either player could win, and give a conjecture on the length of a random game.


Auto STGCN Autonomous Spatial Temporal Graph Convolutional Network Search Based on Reinforcement Learning and Existing Research Results. In recent years, many spatial temporal graph convolutional network (STGCN) models are proposed to deal with the spatial temporal network data forecasting problem. These STGCN models have their own advantages, i.e., each of them puts forward many effective operations and achieves good prediction results in the real applications. If users can effectively utilize and combine these excellent operations integrating the advantages of existing models, then they may obtain more effective STGCN models thus create greater value using existing work. However, they fail to do so due to the lack of domain knowledge, and there is lack of automated system to help users to achieve this goal. In this paper, we fill this gap and propose Auto STGCN algorithm, which makes use of existing models to automatically explore high performance STGCN model for specific scenarios. Specifically, we design Unified STGCN framework, which summarizes the operations of existing architectures, and use parameters to control the usage and characteristic attributes of each operation, so as to realize the parameterized representation of the STGCN architecture and the reorganization and fusion of advantages. Then, we present Auto STGCN, an optimization method based on reinforcement learning, to quickly search the parameter search space provided by Unified STGCN, and generate optimal STGCN models automatically. Extensive experiments on real world benchmark datasets show that our Auto STGCN can find STGCN models superior to existing STGCN models with heuristic parameters, which demonstrates the effectiveness of our proposed method.


Complex Markov Logic Networks Expressivity and Liftability. We study expressivity of Markov logic networks (MLNs). We introduce complex MLNs, which use complex valued weights, and we show that, unlike standard MLNs with real valued weights, complex MLNs are fully expressive. We then observe that discrete Fourier transform can be computed using weighted first order model counting (WFOMC) with complex weights and use this observation to design an algorithm for computing relational marginal polytopes which needs substantially less calls to a WFOMC oracle than a recent algorithm.


Quantitative Programming by Examples. Programming by Example (PBE) systems synthesize an intended program in some (relatively constrained) domain specific language from a small number of input output examples provided by the user. In this paper, we motivate and define the problem of quantitative PBE (qPBE) that relates to synthesizing an intended program over an underlying (real world) programming language that also minimizes a given quantitative cost function. We present a modular approach for solving qPBE that consists of three phases intent disambiguation, global search, and local search. On two concrete objectives, namely program performance and size, our qPBE procedure achieves 1.53 X and 1.26 X improvement respectively over the baseline FlashFill PBE system, averaged over 701 benchmarks. Our detailed experiments validate the design of our procedure and show the value of combining global and local search for qPBE.


Harmonically balanced capitulation over quadratic fields of type (9,9). The isomorphism type of the Galois group G of finite 3 class field towers of quadratic number fields with 3 class group of type (9,9) is determined by means of Artin patterns which contain information on the transfer of 3 classes to unramified abelian 3 extensions. First, as an approximation of the group G, its metabelianization M G G", which is isomorphic to the Galois group of the second Hilbert 3 class field, is sought by sifting the SmallGroups library with the aid of pattern recognition. In cases with order M 3 8, the SmallGroups database must be extended by means of the p group generation algorithm, which reveals new phenomena of groups with harmonically balanced transfer kernels and trees with periodic trifurcations. Bounds for the relation rank d2(M) of M in dependence on the signature of the quadratic base field admit the decision whether the derived length of G is dl(G) 2 or dl(G) 3.


High magnification shadowgraphy for the study of drop breakup in a high speed gas flow. Direct observation of the droplet breakup process in high speed gas flows is a critical challenge that needs to be addressed to elucidate the physical mechanisms underlying the fragmentation phenomenon. Here, we present a high magnification and high speed shadowdograph technique that allows the visualization of this process over its whole evolution and resolves detailed features of the breakup zone. The developed experimental method uses a high speed camera equipped with a long distance microscope. The backlight illumination source is provided by the laser induced fluorescence of a dye solution that delivers short pulses at a high repetition rate. Artefacts resulting from the laser coherence are therefore reduced.


Learning Explicit and Implicit Structures for Targeted Sentiment Analysis. Targeted sentiment analysis is the task of jointly predicting target entities and their associated sentiment information. Existing research efforts mostly regard this joint task as a sequence labeling problem, building models that can capture explicit structures in the output space. However, the importance of capturing implicit global structural information that resides in the input space is largely unexplored. In this work, we argue that both types of information (implicit and explicit structural information) are crucial for building a successful targeted sentiment analysis model. Our experimental results show that properly capturing both information is able to lead to better performance than competitive existing approaches. We also conduct extensive experiments to investigate our model's effectiveness and robustness.


Internal migration and mobile communication patterns among pairs with strong ties. Using large scale call detail records of anonymised mobile phone service subscribers with demographic and location information, we investigate how a long distance residential move within the country affects the mobile communication patterns between an ego who moved and a frequently called alter who did not move. By using clustering methods in analysing the call frequency time series, we find that such ego alter pairs are grouped into two clusters, those with the call frequency increasing and those with the call frequency decreasing after the move of the ego. This indicates that such residential moves are correlated with a change in the communication pattern soon after moving. We find that the pre move calling behaviour is a relevant predictor for the post move calling behaviour. While demographic and location information can help in predicting whether the call frequency will rise or decay, they are not relevant in predicting the actual call frequency volume. We also note that at four months after the move, most of these close pairs maintain contact, even if the call frequency is decreased.


Fast Failures in the LHC and the future High Luminosity LHC . An energy of 362 text MJ is stored in each of the two LHC proton beams for nominal beam parameters. This will be further increased to about 700 text MJ in the future High Luminosity LHC (HL LHC) and uncontrolled beam losses represent a significant hazard for the integrity and safe operation of the machine. In this paper, a number of failure mechanisms that can lead to a fast increase of beam losses are analyzed. Most critical are failures in the magnet protection system, namely the quench heaters and a novel protection system called Coupling Loss Induced Quench (CLIQ). An important outcome is that magnet protection has to be evaluated for its impact on the beam and designed accordingly. In particular, CLIQ, which is to protect the new HL LHC triplet magnets, constitutes the fastest known failure in the LHC if triggered spuriously. A schematic change of CLIQ to mitigate the hazard is presented. A loss of the Beam Beam Kick due to the extraction of one beam is another source of beam losses with a fast onset. A significantly stronger impact is expected in the upcoming LHC Run III and HL LHC as compared to the current LHC, mainly due to the increased bunch intensity. Its criticality and mitigation methods are discussed. It is shown that symmetric quenches in the superconducting magnets for the final focusing triplet can have a significant impact on the beam on short timescales. The impact on the beam due to failures of the Beam Beam Compensating Wires as well as coherent excitations by the transverse beam damper are also discussed.


Reinforcement co Learning of Deep and Spiking Neural Networks for Energy Efficient Mapless Navigation with Neuromorphic Hardware. Energy efficient mapless navigation is crucial for mobile robots as they explore unknown environments with limited on board resources. Although the recent deep reinforcement learning (DRL) approaches have been successfully applied to navigation, their high energy consumption limits their use in several robotic applications. Here, we propose a neuromorphic approach that combines the energy efficiency of spiking neural networks with the optimality of DRL and benchmark it in learning control policies for mapless navigation. Our hybrid framework, spiking deep deterministic policy gradient (SDDPG), consists of a spiking actor network (SAN) and a deep critic network, where the two networks were trained jointly using gradient descent. The co learning enabled synergistic information exchange between the two networks, allowing them to overcome each other's limitations through a shared representation learning. To evaluate our approach, we deployed the trained SAN on Intel's Loihi neuromorphic processor. When validated on simulated and real world complex environments, our method on Loihi consumed 75 times less energy per inference as compared to DDPG on Jetson TX2, and also exhibited a higher rate of successful navigation to the goal, which ranged from 1 to 4.2 and depended on the forward propagation timestep size. These results reinforce our ongoing efforts to design brain inspired algorithms for controlling autonomous robots with neuromorphic hardware.


Reverberant Elastography for the Elastic Characterization of Anisotropic Tissues. We derive closed form solutions for reverberant elastography in anisotropic elastic media by adapting the framework used in electromagnetic theory to treat transverse isotropic materials. Different sample setup geometries are analyzed, highlighting their relevance for both optical coherence elastography (OCE) and ultrasound elastography (USE). Numerical simulations using finite elements are used to validate the proposed solutions in practical cases. OCE experiments are conducted in ex vivo chicken muscle samples for the characterization of in plane and out of plane shear modulus assuming a transverse isotropic elastic model. Additionally, we obtained a generalized geometry independent solution for the isotropic media case, thus unifying previous results for reverberant elastography.


A Novel Memory Efficient Deep Learning Training Framework via Error Bounded Lossy Compression. Deep neural networks (DNNs) are becoming increasingly deeper, wider, and non linear due to the growing demands on prediction accuracy and analysis quality. When training a DNN model, the intermediate activation data must be saved in the memory during forward propagation and then restored for backward propagation. However, state of the art accelerators such as GPUs are only equipped with very limited memory capacities due to hardware design constraints, which significantly limits the maximum batch size and hence performance speedup when training large scale DNNs. In this paper, we propose a novel memory driven high performance DNN training framework that leverages error bounded lossy compression to significantly reduce the memory requirement for training in order to allow training larger networks. Different from the state of the art solutions that adopt image based lossy compressors such as JPEG to compress the activation data, our framework purposely designs error bounded lossy compression with a strict error controlling mechanism. Specifically, we provide theoretical analysis on the compression error propagation from the altered activation data to the gradients, and then empirically investigate the impact of altered gradients over the entire training process. Based on these analyses, we then propose an improved lossy compressor and an adaptive scheme to dynamically configure the lossy compression error bound and adjust the training batch size to further utilize the saved memory space for additional speedup. We evaluate our design against state of the art solutions with four popular DNNs and the ImageNet dataset. Results demonstrate that our proposed framework can significantly reduce the training memory consumption by up to 13.5x and 1.8x over the baseline training and state of the art framework with compression, respectively, with little or no accuracy loss.


Response of Selective Attention in Middle Temporal Area. The primary visual cortex processes a large amount of visual information, however, due to its large receptive fields, when multiple stimuli fall within one receptive field, there are computational problems. To solve this problem, the visual system uses selective attention, which allocates resources to a specific spatial location, to attend to one of the stimuli in the receptive field. During this process, the center and width of the attending receptive field change. The model presented in the paper, which is extended and altered from Bobier et al., simulates the selective attention between the primary visual cortex, V1, and middle temporal (MT) area. The responses of the MT columns, which encode the target stimulus, are compared to the results of an experiment conducted by Womelsdorf et al. on the receptive field shift and shrinkage in macaque MT area from selective attention. Based on the results, the responses in the MT area are similar to the Gaussian shaped receptive fields found in the experiment. As well, the responses of the MT columns are also measured for accuracy of representing the target visual stimulus and is found to represent the stimulus with a root mean squared error around 0.17 to 0.18. The paper also explores varying model parameters, such as the membrane time constant and maximum firing rates, and how those affect the measurement. This model is a start to modeling the responses of selective attention, however there are still improvements that can be made to better compare with the experiment, produce more accurate responses and incorporate more biologically plausible features.


The annealed Calderon Zygmund estimate as convenient tool in quantitative stochastic homogenization. This article is about the quantitative homogenization theory of linear elliptic equations in divergence form with random coefficients. We derive gradient estimates on the homogenization error, i.e. on the difference between the actual solution and the two scale expansion of the homogenized solution, both in terms of strong norms (oscillation) and weak norms (fluctuation). These estimates are optimal in terms of scaling in the ratio between the microscopic and the macroscopic scale. The purpose of this article is to highlight the usage of the recently introduced annealed Calderon Zygmund (CZ) estimates in obtaining the above, previously known, error estimates. Moreover, the article provides a novel proof of these annealed CZ estimate that completely avoids quenched regularity theory, but rather relies on functional analysis. It is based on the observation that even on the level of operator norms, the Helmholtz projection is close to the one for the homogenized coefficient (for which annealed CZ estimates are easily obtained). In this article, we strive for simple proofs, and thus restrict ourselves to ensembles of coefficient fields that are local transformations of Gaussian random fields with integrable correlations and H "older continuous realizations. As in earlier work, we use the natural objects from the general theory of homogenization, like the (potential and flux) correctors and the homogenization commutator. Both oscillation and fluctuation estimates rely on a sensitivity calculus, i.e. on estimating how sensitively the quantity of interest does depend on an infinitesimal change in the coefficient field, which is fed into the Spectral Gap inequality. In this article, the annealed CZ estimate is the only form in which elliptic regularity theory enters.


Toward Forgetting Sensitive Referring Expression Generationfor Integrated Robot Architectures. To engage in human like dialogue, robots require the ability to describe the objects, locations, and people in their environment, a capability known as "Referring Expression Generation." As speakers repeatedly refer to similar objects, they tend to re use properties from previous descriptions, in part to help the listener, and in part due to cognitive availability of those properties in working memory (WM). Because different theories of working memory "forgetting" necessarily lead to differences in cognitive availability, we hypothesize that they will similarly result in generation of different referring expressions. To design effective intelligent agents, it is thus necessary to determine how different models of forgetting may be differentially effective at producing natural human like referring expressions. In this work, we computationalize two candidate models of working memory forgetting within a robot cognitive architecture, and demonstrate how they lead to cognitive availability based differences in generated referring expressions.


Security and Privacy Considerations for Machine Learning Models Deployed in the Government and Public Sector (white paper). As machine learning becomes a more mainstream technology, the objective for governments and public sectors is to harness the power of machine learning to advance their mission by revolutionizing public services. Motivational government use cases require special considerations for implementation given the significance of the services they provide. Not only will these applications be deployed in a potentially hostile environment that necessitates protective mechanisms, but they are also subject to government transparency and accountability initiatives which further complicates such protections. In this paper, we describe how the inevitable interactions between a user of unknown trustworthiness and the machine learning models, deployed in governments and public sectors, can jeopardize the system in two major ways by compromising the integrity or by violating the privacy. We then briefly overview the possible attacks and defense scenarios, and finally, propose recommendations and guidelines that once considered can enhance the security and privacy of the provided services.


A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems. We propose a set of compositional design patterns to describe a large variety of systems that combine statistical techniques from machine learning with symbolic techniques from knowledge representation. As in other areas of computer science (knowledge engineering, software engineering, ontology engineering, process mining and others), such design patterns help to systematize the literature, clarify which combinations of techniques serve which purposes, and encourage re use of software components. We have validated our set of compositional design patterns against a large body of recent literature.


Models of Smoothing in Dynamic Networks. Smoothed analysis is a framework suggested for mediating gaps between worst case and average case complexities. In a recent work, Dinitz et al. Distributed Computing, 2018 suggested to use smoothed analysis in order to study dynamic networks. Their aim was to explain the gaps between real world networks that function well despite being dynamic, and the strong theoretical lower bounds for arbitrary networks. To this end, they introduced a basic model of smoothing in dynamic networks, where an adversary picks a sequence of graphs, representing the topology of the network over time, and then each of these graphs is slightly perturbed in a random manner. The model suggested above is based on a per round noise, and our aim in this work is to extend it to models of noise more suited for multiple rounds. This is motivated by long lived networks, where the amount and location of noise may vary over time. To this end, we present several different models of noise. First, we extend the previous model to cases where the amount of noise is very small. Then, we move to more refined models, where the amount of noise can change between different rounds, e.g., as a function of the number of changes the network undergoes. We also study a model where the noise is not arbitrarily spread among the network, but focuses in each round in the areas where changes have occurred. Finally, we study the power of an adaptive adversary, who can choose its actions in accordance with the changes that have occurred so far. We use the flooding problem as a running case study, presenting very different behaviors under the different models of noise, and analyze the flooding time in different models.


News Cover Assessment via Multi task Learning. Online personalized news product needs a suitable cover for the article. The news cover demands to be with high image quality, and draw readers' attention at same time, which is extraordinary challenging due to the subjectivity of the task. In this paper, we assess the news cover from image clarity and object salience perspective. We propose an end to end multi task learning network for image clarity assessment and semantic segmentation simultaneously, the results of which can be guided for news cover assessment. The proposed network is based on a modified DeepLabv3 model. The network backbone is used for multiple scale spatial features exaction, followed by two branches for image clarity assessment and semantic segmentation, respectively. The experiment results show that the proposed model is able to capture important content in images and performs better than single task learning baselines on our proposed game content based CIA dataset.


Sharp asymptotic estimates for a class of Littlewood Paley operators. It is well known that Littlewood Paley operators formed with respect to lacunary sets of finite order are bounded on L p ( mathbb R ) for all 1 p infty . In this note it is shown that S mathcal I E 2 L p ( mathbb R ) rightarrow L p ( mathbb R ) sim (p 1) 2 quad (p rightarrow 1 ) , where S mathcal I E 2 denotes the classical Littlewood Paley operator formed with respect to the second order lacunary set E 2 pm ( 2 k 2 l ) k,l in mathbb Z text with k l .


Notes on Spaces of Functions Converging at Infinity. This preprint concerns Banach spaces of functions converging at infinity. In particular, spaces of continuous functions, Lebesgue spaces and sequence spaces. In each framework we show versions of Riesz's representation theorem.


Improved Binary Artificial Bee Colony Algorithm. The Artificial Bee Colony (ABC) algorithm is an evolutionary optimization algorithm based on swarm intelligence and inspired by the honey bees' food search behavior. Since the ABC algorithm has been developed to achieve optimal solutions by searching in the continuous search space, modification is required to apply this method to binary optimization problems. In this paper, we improve the ABC algorithm to solve binary optimization problems and call it the improved binary Artificial Bee Colony (ibinABC). The proposed method consists of an update mechanism based on fitness values and processing different number of decision variables. Thus, we aim to prevent the ABC algorithm from getting stuck in a local minimum by increasing its exploration ability. We compare the ibinABC algorithm with three variants of the ABC and other meta heuristic algorithms in the literature. For comparison, we use the wellknown OR Library dataset containing 15 problem instances prepared for the uncapacitated facility location problem. Computational results show that the proposed method is superior to other methods in terms of convergence speed and robustness. The source code of the algorithm will be available on GitHub after reviewing process


Natural Language Processing Advancements By Deep Learning A Survey. Natural Language Processing (NLP) helps empower intelligent machines by enhancing a better understanding of the human language for linguistic based human computer communication. Recent developments in computational power and the advent of large amounts of linguistic data have heightened the need and demand for automating semantic analysis using data driven approaches. The utilization of data driven strategies is pervasive now due to the significant improvements demonstrated through the usage of deep learning methods in areas such as Computer Vision, Automatic Speech Recognition, and in particular, NLP. This survey categorizes and addresses the different aspects and applications of NLP that have benefited from deep learning. It covers core NLP tasks and applications and describes how deep learning methods and models advance these areas. We further analyze and compare different approaches and state of the art models.