We are on the verge of a great shift in scientific thinking. This is because we are finally looking at ‘how’ we think in order to solve problems. The result is that we solve problems by observing partial information and then decide on the best solution. This is called inference. If one can identify a universal method of inference, one should always get the most useful answers based on the information that one has available. This “magic box” may seem like a pipe dream but I think we are making it a reality. Already my recent work unifies two important areas of inference: Bayesian inference and Maximum Entropy inference. This unification cannot be understated. Not only do I show that Bayes and MaxEnt are special cases of this new method and therefore can reproduce every aspect of Bayesian and MaxEnt formalism, but now I am able to tackle problems that we were not able to solve until now. My current research entails using this method of inference (which we call ME) to solve problems in ecology, economics, complex systems, non-equilibrium systems, physics and more. In short I hope to unify scientific thought by discovering and probing the fundamental principles of inference in order to create a universal method. With this method we will be able to answer any question, regardless of what discipline the information comes from.

Below is a list of some fields that are of particular interest to me:

Econometrics – I am interested in solving problems here that cannot be solved with current methods. I wrote a paper in which I examine a large deviation solution to a toy problem in econometrics. The paper consists of applying the ME method to the problem and showing that the large deviation theory solution is a special case of the ME solution. Therefore we can now say that Sanov’s theorem is a special case of ME, thus illustrating further that ME is universal. This paper was presented at the 3rd Econophysics Colloquium in Ancona, Italy. I was also invited to present an extended version of this paper to the 6th Advances in Econometrics conference in Baton Rouge, LA. I would like to expand this line of research by applying these ideas to more structured problems.

Ecology – This is increasingly becoming an important field of study. I wrote a paper that showed the improper use of Shannon’s entropy as a diversity measure in ecology. However by using ME to define the measure, one gets a more accurate measure and one can now include information that relates the species being measured. This is a breakthrough in the field. This was presented at NECSI in October. I would like to apply this new definition of diversity in many areas. Not only to help understand species diversity, but diversity in social sciences and biology (evolution, genomes, etc) as well.

Complexity – I am interested in studying what makes problems complex and how to simplify them. I think applying the ideas from inference will help solve these issues. To explore this, I wrote a paper where several agents each receive partial information about a system in the form of data. In addition, each agent agrees to, or is informed that there are certain global constraints on the system that are always true. Each agent is then asked to make inferences about the entire system. The point of this paper is to show how different agents predict different outcomes following the new model. The system becomes more complex as we add agents and allow them to share information. This system can have a geometrical form, such as a crystal or molecular structure. The shape may dictate how the agents are able to share information, such as sharing with nearest neighbors. This model can be used to examine emergent properties of the system, its evolution, anything. This was presented at NECSI in October. This particular model seems suited for all of the topics mentioned in this document.

Non-equilibrium systems – I was exploring the idea by applying the ME method to a simple physics problem where one knows the equilibrium condition (a constraint) yet is able to take some data as well in order to better approximate the actual ‘state’ of the system. Essentially this approach allows us to use both macroscopic variables and microscopic ones to give us the best estimation of what we seek. Since then I have been approached by many people with different problems in different disciplines asking to collaborate with them using this method which I would very much like to do.

I am also extremely interested in the concept of reversibility. The idea that we can never ‘go back’ fascinates me. So much so that my oral examination was entitled, “Demonic Computation: Reversibility in Classical Computers.” I would like to pursue the inference approach to defining irreversibility. Here are two papers I am currently working on regarding this idea:

The first is what I call, “The Bayesian Demon" where I discuss the ideas of Landauer's Principle from the Maxwell/Szilard Demon's view point. If one approaches this from a Bayesian point of view, uses the proper entropy (ME) and understands that the minimum amount of observations one can make is ‘1’ observation, then the entropy associated with that observation is exactly klog2 which is what Landauer’s principle predicts. Thus I think I can generalize Landauer’s principle by using purely informational arguments and without assuming that the states are ‘equally likely’ ala Boltzmann. Additionally, the idea of information conservation comes out of this. I would also like to apply this idea in biology where the concept of information storage is often used.

Another paper I am working on is called "Irreversible Chaos: The Path to Complexity and Randomness". I use information geometry (IG) in a similar way to a previous paper that I co-authored last year where we examine two Gaussians with IG. In it we show that when the Gaussians are uncorrelated one gets linear results. When they are correlated, one gets results that resemble a chaotic system. The novel part of my new paper is that I include a stochastic variable in the beginning which shows up as a Lyapunov component. So, when there is no correlation, the system is linear. No correlation with stochastic variable: simple patterns. Correlations: chaos. Correlations with a stochastic variable: randomness. I think that this idea will be useful in a similar way that Wolfram claims his cellular automata are useful. The major difference here is that by using IG, these statistical systems can model physical systems.

In summary, I think that this new way of thinking, that all information is processed the same way will revolutionize not only the fields in which it is applied, but the way we approach any new question in general.