Thursday, August 6, 2020

Our latest publication: "Just Don’t Fall: An AI Agent’s Learning Journey Towards Posture Stabilisation"


Our latest publication has been chosed as one of the featured articles on AI journal
It takes you through the journey an AI agent takes to learn to stand still. The paper touches on the intertwining of reinforcement learning and biomechanical modelling to advance our understanding of human movement.

Hope you have a read here and give me you feedback.

Monday, March 30, 2020

What our eyes reveal? Part I Our Pupils

They say,
 "Beauty lies in the eye of the beholder".
But after four years studying human eye movement, I feel more inclined to believe that,
   "Information, a lot of information, lies in the eye of the beholder.

Encoded in the way our eyes move is a lot of information that reflect our mental and psycological state. Most of these movements are involuntary; we cannot prevent them from happening. My PhD. research work has led me to read many studies investigating the relevance of different eye movements.

Pupil dilation (the pupil becomes bigger) and restrictions (the pupil becomes smaller) is one of those involuntary movements that reflects different mental and psychological states. The pupil's main function is to regulates the amount of light entering the eye. That mean that when we are looking at a scene where so much light is reflected unto our eyes, out pupil restrict. However, looking at a darker scene makes our pupils dilate to increase light intake for better vision.

However, other events can have an effect on our pupils. the fascinating thing is that most of these effects are involuntary. Lesion and drugs can cause a huge change in the pupil size which can be easily observed. Yet, very subtle changes to the pupil can happen due to external sensory events like fear, or with emotional and mental processes. 
In one of my papers [2], we investigated the efferct of the perception of risks while driving on the pupil diameter. And we found it can be a metric to determine whether a driver saw a hazard or not.

In another blog I might talk more about the different eye movements and what is hidden behind them.
If interested I have added below a list of related papers. 





[1] Iskander, J., Hossny, M., & Nahavandi, S. (2018). A review on ocular biomechanic models for assessing visual fatigue in virtual reality. IEEE Access6, 19345-19361.
[2] Iskander, J., Hanoun, S., Hettiarachchi, I., Hossny, M., Saleh, K., Zhou, H., ... & Bhatti, A. (2018, April). Eye behaviour as a hazard perception measure. In 2018 Annual IEEE International Systems Conference (SysCon) (pp. 1-6). IEEE.
[3] Iskander, J., Jia, D., Hettiarachchi, I., Hossny, M., Saleh, K., Nahavandi, S., ... & Hanoun, S. (2018, October). Age-Related Effects of Multi-screen Setup on Task Performance and Eye Movement Characteristics. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 3480-3485). IEEE.
[4] Iskander, J., Attia, M., Saleh, K., Abobakr, A., Nahavandi, D., Hossny, M., & Nahavandi, S. (2019, October). Exploring the Effect of Virtual Depth on Pupil Diameter. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) (pp. 1849-1854). IEEE.

Sunday, January 26, 2020

Visual Fatigue and Ocular Biomechanics

In 2016, about 6.5 million virtual reality devices were shipped. By 2020, the market size of the virtual reality software and hardware is estimated to reach $40.4 billion USD.  

Virtual reality is known to most of us as a gaming tool, entertainment only. But, it's much more than that, indeed our life is soon turning into a huge virtual game. Virtual reality is an important aspect of military training, pilot training and even in medical staff training. It is also a part of our children's education and development. It is a mean of remote communication and knowledge transfer. It's growing rapidly and with time we will depend on it more and more like every other technology ever invented and changed our lives forever, the telephone, Internet, mobile phones and smart phones. 

But what are the side effect of being immersed in virtual environments for long periods of time? Are they safe? Can we continue our lives safely after using them? Can we drive our cars? Operate heavy machinery safely?  These questions are still to be answered. However, those of you who used any kind of virtual reality devices may have experienced nausea, dizziness, double vision or headaches. These are due to the conflict between the real world and the virtual world. Our brain accepts conflicting signals from our eyes. 

Our eyes are tightly coupled to our mental state and changes in our eye behaviour says a lot about our quality of experience and level of fatigue. The discomfort experienced in virtual environments is mainly due to mental overloading which can be seen in change in eye behaviour and can never happen intentionally. 

My work is to use our eye movements and the biomechanics of our vision systems to better design virtual environments devices.  

Till now, there are no comprehensible complete eye models that contain all behavioural and anatomic properties of the eyes and the eye muscles. That’s why we developed our model. This here is our model. It can simulate different eye movement systems from slow pursuit movements to very rapid saccadic movement that can reach 900 degree/second and also eye-head-neck coordination movement. 

We aim to use it to test virtual environments and calculate the discomfort of the experience. This will be used in modifying the virtual environment design and test it again and so on. 

We use our model to better understand our eyes and mental behaviour in virtual environments and thus to create healthier virtual environments and safer experience for our children and ourselves. 


For more details read my published papers:
[1] Iskander, J., Hossny, M. and Nahavandi, S., 2018. A review on ocular biomechanic models for assessing visual fatigue in virtual reality. IEEE Access, 6, pp.19345-19361. 
[2] Iskander, J., Hossny, M., Nahavandi, S. and Del Porto, L., 2018. An ocular biomechanic model for dynamic simulation of different eye movements. Journal of biomechanics, 71, pp. 208-216.
[3] Iskander, J., Hossny, M. and Nahavandi, S., 2019. Using biomechanics to investigate the effect of VR on eye vergence system. Applied ergonomics, 81, p.102883.

Sunday, December 1, 2019

Notes on “The Media Equation” by Reeves and Nass

The Media Equation, written in 1996  by  Byron Reeves and Clifford Nass

Next are the notes I took while reading the book.
Some parts might look very obvious to us now in 2019. It was, however, an interesting book and can be considered a different yet beneficial introduction to human-machine interaction (HMI).

I think that this equation can be easily mapped to robots and artificial intelligence. 

I have also found this youtube interview with the authors (https://youtu.be/26BclMJQUwo)

Here are my notes


1- Introduction: The media equation
Media = Real life

It is not intuitive. People respond socially and naturally to media even though they believe it is not reasonable to do so, and even though they don’t think these responses characterise themselves. 

Social and natural responses to media are not conscious.

Perception alone can alter a broad range of human behaviour.
Acceptance of what only seems to be real, even though at times inappropriate, is automatic.

2- Media and Manners: 
  • Politeness
“When people ask about themselves, the answer will be more homogeneous and more positive than when someone else asks the same questions.”
The participants in the experiment denied that they had been intentionally polite to a computer. Instead, the responses occurred without conscious awareness. (Automatic and mindless). 
It is possible to apply finding from social and psychological studiesto human-media interaction.
Grice’s Maxims conversation should be guided by 4 principles. 1. Quality ( telling the truth), 2. Quantity (politeness over brevity, yet too long is not acceptable ). 3. Relevance 4. Clarity (customers may trade precision for simplicity. 
It is Polite to Look at people when speaking.
It is Polite to Match Modality: Answer a letter with a letter.
When focusing on comparison, respondents are not thinking about what the polite response should be. This should encourage truthfulness.

  • Interpersonal Distance
When viewers see a picture of a person who appears close rather than far, their evaluations of the person in the picture will be more intense.
Viewers will pay more attention to pictures of people who appear close rather than far.
Pictures of people who appear close will be remembered better than will pictures of people who appear far away.
Pictures can appear near or far by:
1- Viewing distance
2- Perceived distance using screen size
3- Camera shot (close up/ wide)

  • Flattery
People will believe that they did better on a task when they are flattered by a computer than when the computer doesn’t give any evaluation.
People will like the computer more and believe it did a better job when they are flattered by the computer than when the computer says nothing about their work.
Whether praise is warranted or not will have no effect on what people think about the praising computer.
People will believe that they did a better job when the computer criticizes them without basis than when the criticism is warranted
People will like the computer more and think the computer is better when it praises then than when it criticizes them.
The psychology literature says that flattery will work best with people who are less confident about their performance. 
Flatter novice quite broadly
Experts should be complimented more subtly by picking out more intricate material and by noticing detail.
All people love to be praised and hate to be criticized.

  • Judging Others and Ourselves
A performance praised by a computer will be perceived as superior to a performance criticised by a computer.
When a computer praises itself, the praise will be perceived as less valid than when the praise comes from an independent computer.
A computer that praises another computer will be liked more than a computer that praises itself.
A computer that criticises another computer will be liked less than a computer that criticises itself.
A computer that criticises another computer will be perceived as more intelligent than a computer that praises another computer.

Social responses to media, while powerful, are not obvious to the people who are being social.

3- Media and Personality
  • Personality of Characters
All of the personality adjectives can be neatly organised into 5 basic dimensions. (The Big Five)
1- Dominance and submissiveness
2-Friendliness(friendly, warm, and sympathetic) and Disagreeability (cold, hard-hearted)
3-conscientiousness, 
4-emotional stability, and 
5-openness
The two most important categories of media personalities are dominance/submisiveness and friendliness/unfriendliness.
Media personalities are readily identified by the peple who experience them.


  • Personality of Interfaces
People will perceive a computer that uses dominant text as having a dominant personality, and a computer with submissive text as having a submissive personality.
Dominant people will say that the dominant computer is more like them than the submissive computer, and submissive people will recognise the submissive computers as more like them than the dominant computer.
Dominant people will prefer the dominant computer, while submissive people will prefer the submissive computer.
To make a computer express dominance/submission
     - Language (assertive/commands versus questions/suggestions)
     - Level of confidence shown on a scale
     - Sequence of interaction (dominant went first always).
     
The creation of personality does not depend on virtually real representations. Our old brains automatically extrapolate when given a little hint.
Virtually all interfaces have a personality. (language in error messages, user prompt, navigation options even font and layout)
Personality of media should be consistent and strong

  • Imitating Personality
People who change to make others happy are preferred over who always try to make others happy.
People like to gain something rather than always have it. (Gain theory)
Imitation is the sincerest form of flattery. If you want to make people feel good, change for them, and change in their direction.
Dominant people will prefer a computer that starts out submissive but then becomes more dominant more than a computer that is consistently dominant.
Submissive people will prefer a computer that starts out dominant but then becomes more submissive more than a computer that is consistently submissive.
Media should adapt to the personality of the user.

4-Media and Emotions
-Good versus Bad
The two hemispheres of the human brain have evolved to be specialised processors of positive and negative information. Positive experiences become associated with the left hemisphere (favors verbal and analytical processing), and negative experiences with the right hemisphere (favors spatial and non-verbal processing).
People can identify a smiling face when it is flashed to the left hemisphere than when it is flashed to the right hemisphere. The opposite occurs for pictures of frowning faces.
Good versus bad is a primary evaluation of mediated experience.
There is more activity in the left hemisphere for material that is evaluated as good, and more activity in the right hemisphere for material that is evaluated as bad.

When psychologists confront a complex question (concerning effect of media), they try to decompose stimuli, breaking things down into small parts. They can then test each one separately.  This has a big disadvantage that it yields something that doesn’t look anything like real life.



-Negativity
The law of hedonic asymmetry, this asymmetry refers to the unevenness of good and bad, notably the dominance of negative experience. There is pleasure that we all get used to, but pain is a different story. We dwell on it.
Negative information demands more attention because it is consequential.
Pollyanna Effect==> a hope that people with some effort can remember positive experiences better than negative ones.

People will dislike negative media
People will pay more attention to negative media than to positive media
People will remember negative media better than positive media
People will have better memory for information that comes after negative media (proactive enhancement) than for information that comes after positive media.
People will have worse memory for information that comes after negative media (retroactive interference) than for information that comes before positive media.

The quicker people are at making decision, the more accessible the information.
Negative information produce better memory for pictures and sound.



-Arousal
Both negative and positive experiences share the potential to arouse. Like valence, arousal has implications for the way people evaluate their experiences, for what people remember and for whether they pay attention.
Arousal and valence together are two basic dimensions of emotion. They are independent of each other. And knowing one, won’t predict the other.
High states of arousal increase the vigilance with which people attend to the world and lead people’s processing systems to work at full capacity.
People will respond to media content using the same basic dimensions of emotion (valence and arousal), that they use when responding to real-life experiences.
Highly arousing material will be better remembered than less-arousing materials.

5-Media and Social Roles
-Specialists
Why are labels (ex. “Specialist”) so powerful? 1-categorisation simplifies interaction. Applying labels provides a predictability that people are reluctant to scrutinise the label’s accuracy. 2- people are biased towards acceptance because it requires no work. Doubt requires greater thought.

Content on a tv set labeled a specialist in that type of content will be perceived as superior to identical content on a tv that is labeled a generalist.

The label “specialist” was powerful enough to influence assessment not only of content but also of visual quality.
All responses were unconscious and automatic.
Image.jpeg
Specialization influences feelings about media as well as objective judgments about the information that media presents.


-Teammates
Teammates are identified by two factors: group identity(a marker or just a name) and group interdependence (behaviour of one member affects all other team members).
People teamed with a computer will feel more similar to the computer than people who are not teamed.
People teamed with a computer will think better of the computer than people who are not teamed.
people teamed with a computer will cooperate more with the computer than people who are not teamed.
People who are teamed with a computer will agree more with the computer than people who are not teamed.
Interdependence had an enormous effect and identification had a less powerful effect esp when combined with interdependence.
Computers and users should be peers, and people should be made to feel dependent on computers without feeling inferior or superior. Being on the same team encourages people to think that the computer is more likeable and effective, and it also promotes cooperation and better performance.


-Gender
Gender can be applied to media through 1) presentation of actual people. 2)using voice

Evaluations from male-voiced computers will be taken more seriously than evaluations from female-voiced computers.
A female-voiced computer that evaluates will be liked less than a male-voiced computer that evaluates.
Female-voiced computers will be seen as more knowledgeable about love and relationships than will male-voiced computers.
Male-voiced computers will be seen as more knowledgeable about technical subjects than will female-voiced computers.
People will assign higher levels of drive to people depicted with masculine voices than to those with feminine voices.
People will assign higher levels of extroversion to people depicted with masculine voices than to those with feminine voices.
People will assign higher levels of intelligence to people depicted with masculine voices than to those with feminine voices.
Evaluations from media people depicted with masculine voices will be taken more seriously than evaluations from people depicted with feminine voices.

Viewers applied stereotypes simply on the basis of a subtle gender cue in voice, even though all of the voices came from female faces and audio speakers sitting beside a television screen.
People try to know the gender of those they interact with because it cues them about how to behave, what to say, and how to say it. 

-Voices
One presence per voice, not one presence per machine
Users will respond to different voices on the same computer as if they were different social actors.
Users will respond to the same voice in different computers as if it were the same social actor.
The male voice tutor that was praised by a different male voice was perceived as better than the male voice tutor that praised itself. The female-voice tutor was praised by a different female, was perceived worse than the female-voice tutor that praised itself.  
The number of social actors in the exchange equals the number of voices that enter the conversation, not the number of machines or the visual similarity between them.
Voices carry baggage, they activate stereotypes associated with gender, age, personality, and other things.

-Source Orientation
Psychological research shows that people automatically assign responsibility for messages to those who deliver them.

The computer will be considered the source of information, not the programmer.
People working with computers will not think about programmers during an interaction
Interactions with computers are more desirable for users when they don’t think about a programmer.

How to make people think about the programmer who wrote the code, is by asking the people to do so,
People do not respond socially to computers because they imagine an interaction with a programmer. Computers are no different from traditional media: Presence is relevance.

6-Media and Form
-Image Size
Size is one of the most primitive cues we have about what’s happening in the environment. Size can determine whether the objects and people around us are safe or dangerous, something to approach or reason to flee.
Size also influences arousal, presumably because large things need attention.
When displays are large, more of the picture is in peripheral vision. This means that the boundary between the screen and the rest of the environment is farther in the corner of vision. This makes the boundary less noticeable, a feature with important implications for arousal.
Action outside of direct attention could be more consequential than action right in-front of us, the response to motion in peripheral vision is greater arousal. The preparation for response is more urgent.

Larger pictures will be more arousing than smaller ones.
Larger pictures will be better remembered than smaller ones.
Larger pictures will be better liked than smaller ones.

Too much arousal can be counterproductive in learning. It is quite possible for emotional experience to distract. The consequence is that little mental energy is left to contemplate, rehearse, or integrate information with prior experience.
Arousal predicts memory; when arousal is high, the brain marks the event that precipitated the arousal, and we are more likely to remember it. It is mostly information specifically related to arousal, and not the information that most people would think of as educational.

-Fidelity
The psychology of human vision suggests that sharper pictures solve no important visual problems, as low visual fidelity is familiar and not particularly troublesome. 
Lower fidelity visual images on the screen will be evaluated no differently than higher-fidelity images.
Audio fidelity:
Audio fidelity will affect attention to media
Audio fidelity will affect people’s memory for audio information
People will evaluate better audio fidelity differently than poorer audio fidelity.

When it is poor, presentations sound unnatural, and people consciously monitor the content. When it is good, people are immersed.

-Synchrony
Audio-video asynchrony will result in negative evaluations of the speaker.
There was no psychological discount for broken or underdeveloped technology. Rather, technical problems have social consequences.

-Motion
When a picture presents a visual surprise, viewers will orient to the surprise
When objects or people in pictures move, attention will be higher than during segments with no motion.
Attention (measured through EEG) is strongly associated with the presence and absence of motion
Several studies in psychology show that learning improves when there are visual rests. For example, students learn more when there are large windows in a classroom because they can take visual breaks. Memory is enhanced when people have a chance to stop and think about information instead of being forced to continually orient to new stimuli.
Motion in the peripheral vision is more arousing than motion at exactly the point of visual focus. The evolutionary significance of this is easy to see. Motion that we aren’t directly looking at is potentially more harmful than motion that we stare at.

-Scene Changes
Visual Cuts in media will cause a visual orienting response (jump in attention and an acute psychological focus on what is new) in viewers. The most pronounced effects didn’t occur immediately, however. The greatest effects were one second after the cut; they dissipated quickly thereafter.

Cuts between segments that are semantically related will be less intrusive than cuts between segments that are semantically unrelated.

The number of cuts and the level of attention are related. More cuts were given less attention than those that had none. Media can easily exceed processing capabilities. When this happen, people tune out.

Visually dynamic messages will create more favourable evaluations of people in them than visually static messages. More cuts equaled more honesty, trust, intelligence, open-mindedness, and sincerity.

After long periods of exposure to a media segment, interruptions will require more attention than when exposure to the original material is shorter.

Long interruptions, regardless of the length of exposure to the original media segment, will require more attention to the original material after the interruption.

-Subliminal images
Subjective experience, however, is often wrong. 
Images in television messages that people cannot consciously identify will influence how people make judgements about media.
Quickly flashed images that conveyed emotions became active in viewers’ minds, even though the viewers had no subjective awareness of their presence.
A patterned mask (black and white geometric shapes) is used to avoid the linger of the image on the retina longer, such that viewer has time to identify and consider the image.


Conclusion:

People’s response to media are fundamentally social and natural.
Media are more similar than different.
The Media Equation is automatic (unconscious)
Perceptions are far mor influential than reality defined more objectively.
Humans are built to reduce complexity, not only to create it. 
Simple is also more predictable