Last month I listened to Hillary Clinton talking about resilience. That gave me the chance to think about how I find resilience important in research and in life in general (doing research with three kids just requires resilience by default :)). And why I also like to build resilience in my kids.
I think that you really have to be resilient if you want to succeed in research. There will always be challenges that you have to face. There will be experiments that go wrong, rejected papers, or someone publishing your ideas before you. You never know what will be the world in next week, next month, next year. To manage this world, you have to believe in yourself and your abilities, know what is your goal, learn how to ask for help, and find always new directions for research.
So what the resilience actually is? Resilience refers to one’s ability to survive and flourish despite the risk, to adjust and adapt to changes as they occur, and to recover from misfortunes. As such, it covers various personal features such as self-belief (confidence in your own abilities), optimism (the belief that you will experience good outcomes in your life), purposeful direction (to which extent you have clear goals), adaptability (how much you are willing to adapt your behavior/approach in response to changing circumstances), ingenuity (how much you perceive that you are capable of finding solutions to problems), challenge orientation (how much you enjoy experiences that challenge you), emotion regulation (ability to remain calm in stressful situations), or support seeking (how much you are willing to ask for help and support in difficult situations).
Here are 10 tips from me on how you can support resilience during your research career:
🎯Be organized. It might sound simple, but I have a feeling this is where success really starts. At least for me. And after seeing many many students, also for many others. Having always a list of tasks you want to work on, have your results and data saved and reusable, and have noted down your ideas. This all gives you a lot of self-confidence and calmness. Someone asks you to remake experiments or recompute something? You know you can do it. Does a new task come in? You don’t lose track of other tasks. You know what you can afford to postpone, what you have to prioritize. Find your way to keep track of your life. I like to use Trello, a paper list of the current tasks, and a paper notebook to make notes from meetings or note down my ideas.
🎯Finish your tasks. Do you also like the feeling that comes after you managed to submit a paper before the deadline, defend successfully your project, or after you finished slides for the lecture before the lecture itself? I enjoy it a lot. It boosts my confidence in stars. There is nothing better to build your resilience than the feeling that you can manage your work and tasks. Managing tasks does not mean being perfect. You have to learn what is “good enough”. Try your best to make it as good as possible, but don’t give up and finalize it.
🎯Find easy wins. To get the feeling that you can succeed, it really helps to find places, where it is easy for you to win. Start with a local conference, with workshops on good conferences, or join the team (not being the first author) that submits a paper to a great journal/conference to learn what is the required quality of the paper and what their work looks like.
🎯Challenge yourself. Don’t be satisfied with the easy wins. Small successes will encourage you and are really important, but try always to reach for something better. Who said you cannot publish in Nature? Maybe not yet, but you can aim for it. Always find a bit better journal in your field than last time, try to reach out to the great researchers in the field and start collaborating with them, or try to ask companies for collaboration. Maybe some will not answer, maybe you get rejected, maybe you are not ready yet. But maybe they will answer and give you a chance.
🎯Try again and again. Knowing “what is good enough” is something you have to learn by trial and error. You have to try to submit your papers to several journals and conferences to find out what is good enough for the given journal. To know what is already worth to be published. Proficiency comes from practice. Writing is just another skill that you have to learn. Trying, again and again, will help you to build resilience. You will improve your communication skills, you will learn what people around want and how you can help them.
🎯Be prepared. I find out, that it really helps my confidence when I am well prepared. If I am preparing for teaching, presenting my work at a conference, talking to potential collaborating companies, or even to students. Preparing yourself, trying your presentation in advance, studying the materials, and trying to ask yourself the questions they might ask. All that helps me to get confidence and avoid a lot of stress from unexpected questions.
🎯Learn to work smart and hard. I really like to use my energy and power efficiently (see the crow on the front image). With three kids, it is almost a necessity. You have to learn how to read your papers on playgrounds, how to make meetings while the small baby is sleeping on your lap, and how to work at night. Kids also teach you, that you don’t have infinite time for your tasks, so you have to accept imperfections. Don’t be lazy, work as much as you can, but also learn how to recharge your energy, how to relax, and how to reuse what you already have. For me, it really works a lot to be well slept and have enough sports activity to deal with stressful situations and negative feedback. Not that it would be easy, but it is easier.
🎯Rejection is just feedback. I think that this cannot be repeated enough times. Reviewers are (most of the time) not bad guys. They try their best to give you valuable feedback. If they rejected your paper, it might mean that you didn’t present well your contributions, that your work is not ready yet, or maybe you didn’t well compare it to other works in the field. Take it, try to learn from it, evaluate what is valuable in the feedback, improve your work and submit it as soon as you can again.
🎯Give yourself deadlines. There is a sad story about one grant proposal, that I work on already for almost two years. The problem is, that that type of grant can be submitted anytime. It has high potential, and the topic is interesting. We submitted several proposals in between and some were already successful. But this one has no deadline and is still on the table. It is really hard for me to work without deadlines. Maybe you have it as well. The problem is, that when you have undone things, it feels very similar to the unsuccessful ones. If anyhow possible, find grants, conferences, or journals with deadlines. If there are no deadlines, try to set your own deadlines. Maybe give yourself some presents, if you make it to the deadline :).
🎯Seek support. One of the biggest problems I see with my students is that they don’t ask questions. They don’t want to show me the undone work. But when being in trouble, help is the most valuable. Also by explaining your problem to someone else, you will get way more clarity about the problem. Ask Google and if you don’t get your answer, seek help so that you are not wasting unnecessary time.
The first run of the online course “Research and Publish with ease” just finished and I am still full of thoughts and feelings. It was a very nice experience. The course was running for 4 weeks (21.2.-20.3. 2022) in a closed LinkedIn group. Before I forget how it was going, I wanted to write down some insights.
I promised the participants that in the course, they will: 🪶 clarify their goals in the research 🪶 learn how to present their contributions 🪶 practice how to present their research to the public 🪶 learn how to write and structure the paper to please the reviewers 🪶 learn about the criteria they should care for to become a senior researcher
Biggest takeaways of the participants
Based on their main takeaways from the course, the participants did learn all of that and also something on top. See the list below:)
How did I promote the course?
I was promoting the course on Facebook and LinkedIn, and sending the invitation to researchers I know around the world to share the information with their students. In the end, 81 people enrolled, and from that 60 joined the LinkedIn group.
It was a real challenge for me to get in contact with people, motivate people to join the group, and make them actively participate. Being many of us deadline hunters also didn’t help :-D.
Who were the participants?
Before I started the course, I wanted to know which country were the participants from and what type of research they are doing. As you can see below, most of the participants were from CR, followed by UK, Slovakia, and Germany. Altogether participants from 20 countries were involved.
As for the field of research, the most participants were in Robotics, Engineering (including electronics and electrical systems), Psychology and social science, Economics, and Material science. Anyway, the range of the research topics is really big covering fields like Economics, Marine science, Natural language processing, and Architecture.
How was the course structured?
I prepared for every week on Monday prerecorded materials (approx. 30 minutes) that I uploaded on LinkedIn. On Thursday we had a live Zoom call for answering questions and digging deeper into the topics, followed by live breakout rooms, that the active participants really enjoyed as they could train with other people to present parts of their research and get feedback.
I was running the course in a LinkedIn group and it worked quite fine. For better organization of the materials, I might the next time consider some other platform. Also, LinkedIn has a restriction that you can upload a maximum 15 minutes video – not that great for tutorials. Next time I will definitely put the videos on YouTube and just link them. StreamYard live streaming to LinkedIn sounds also like an interesting option.
How did the participants like the course?
How did the course meet the expectations of the participants?
Finally, I would like to share some of the participants’ reviews with you:
I am happy, that the participants liked that “the course will equip you with the essential tools to tackle publishing from A to Z”, gave them “useful tips” in “short, well-structured tutorials” and found it “systematic”. What made me the happiest was, reading in the feedback, that they “got confident, motivated and inspired to navigate my research” or “get the confidence to put yourself out there”. Because that was one of the underlying goals of the course.
I was a bit unsure if to make the course for all of the fields of research. But I decided to give it a try. It seemed, that putting together people from different fields and stages of research was beneficial and people appreciated the interaction a lot: “The best thing about it is the community and the experience shared on it.” or “The opportunity to present your work to others and getting feedback is very useful!” Nevertheless, as with everything, it had some negative sides. For example, in breakout rooms, it was sometimes difficult (especially for younger researchers) to give feedback to other participants if they were from totally different fields.
Improvements? Next run? My takeaways.
I have to say, that I was quite happy with how the course went, that people exchanged information and that I also already see some new collaborations and connections between people created (although I hoped for way more, but let’s keep it for next runs :). I just have some small improvements I want to do for the next rounds, like improving the sound of the videos, and adjusting timing as well as some of the assignments for interactive sessions in breakout rooms to make sure that people can always present their research and also get valuable feedback.
If you are interested in the next run of the course, or you know anyone who might benefit from it, you can sign up for the waiting list here: Sign up for the waiting list. I plan to run the course most probably again in the summer (June-August) but have not yet decided on the exact dates.
My three main takeaways are:
I really enjoy to work with motivated smart people and share the knowledge with them. I love the motivation I get from these interactions.
Focusing on one thing at one time really helps to achieve whatever you want.
Anytime you feel demotivated or hopeless, just go on! The diamond might be just one dig further…
Already looking forward to the next run of the course!
I am happy to announce that this year, we got accepted another 3 papers (1 journal and 2 conference papers) which deal with robot self-calibration.
This long term project is running under great supervision of Mgr. Matej Hoffmann, Ph.D. (see his website) who focuses on how babies develop the sense of their bodies and space around it and how self-touch might contribute to it (GACR project, EXPRO project). This connects closely also to self-calibration of human who might learn about kinematics of their bodies via touching its individual parts.
In our efforts we try to transfer these ideas to the robots and explore if the usage of this new sensorical modality might be useful for their calibration. We focus mainly on humanoid robots, but explore also the usefulness of these methods for the calibration of industrial robots.
Our long journey was this year completed by 3 papers – 2 conference papers at Humanoids conference and 1 journal paper in Robotics and Computer-Integrated Manufacturing (RCIM) journal.
Here I am. A small confused boy. My mum is Czech and my daddy is German. Together they speak most of the time English. My two big sisters are Czech who understand and speak some English, but with me, they speak mainly via smiles. To make it even worse, my uncle is Czech, but my aunt is Spanish. Their daughter speaks a weird Spanish-Czech language. Other aunt is German, and her boyfriend Portuguesse…. It is really difficult to understand this world and people in it. For now, I am mainly silent. Observing and listening. Laughing a lot and sometimes screaming when I get frustrated. People sometimes say that I am a poor guy and that I have it tough. But my mum and dad say that I do not have to worry, that I will learn to understand them as quickly as other kids and on top, I will be able to speak with more kids around the world.
Every year, there is an increased number of marriages of couples speaking different languages. On top of that, add immigrants, and people who just want to teach their kids a foreign language as soon as possible. Raising a kid in bilingual or trilingual settings is not anymore the specialty of Switzerland. Did you know that Tucanoan people (in Amazonia) are all native bilinguals? How is that possible? They practice the so-called linguistic exogamy rule, which means, that each marriage has to happen outside the linguistic community .
Maybe you have the same feeling as many other people, that kids in bilingual or trilingual families have it harder. How many times have I heard the sentence “Oh, what a poor guy,” which was trying to point out how hard it is for our son to learn a language in our trilingual family. The old belief that speaking to kids in different languages will negatively impact their learning and understanding of language is not valid anymore. Today we know, that not only that they are able to manage to easily distinguish between languages and learn with the same speed as monolinguals, but also that multilingualism affects the structure and functioning of the brain, having a positive effect on cognitive processes as well as protecting the brain from the cognitive decay.
The belief that multilingualism has negative impact on language acquisition was based on the belief that for the infant, all the languages are part of one language and therefore they are only able to separate them after establishing initial lexicon and syntax . But already in 1962  an experiment showed that bilingual children outperformed monolinguals in both verbal and non-verbal intelligence tests. From the recent research it seems that infants are able to separate languages right from their birth using various approaches. The native speakers can distinguish native language from another language which is in different rhythmical classes already at birth  and the ones from the same rhythmical class at 4-5 months. Bilingual kids can do the same for both of their native languages. It seems that distinguishing between various languages is way easier for bilingual infants than it was believed. They use visual cues, phonological and lexical differences, as well as a specific stress pattern of the individual languages.
In one of the studies, they found out that bilingual newborns are able to distinguish various languages from each other, even when the speakers are muted. It seems that what they hear and how they process this information differ from monolingual kids. Both native monolinguals and bilinguals can distinguish the native language only based on visual cues at 4 months , but the learners who learned the second language later in life, never get the same skill and score only at chance level. Seemingly, bilingual children are more sensitive to even small facial distinctions as they keep a very good level of discriminating the native language among languages, keeping the skill to discriminate the native language only based on the visual cues during the lifetime.
Bilingual children are also able to distinguish languages using phonological and lexical differences better than monolingual kids.  For example, one study showed that bilingual children are able to distinguish Spanish and Catalan vowels not only at 4 months as monolingual kids do but also at 12 months. Also, Japanese adults who were raised as bilingual or learnt English early can distinguish between English “r” and “l”, which is not possible for late English learners.
Another help for distinguishing languages is a stress pattern. For every language, there is a different stress pattern and the native stress pattern helps you to segment words from each other, which is crucial for language acquisition. If a kid is learning only one language, it learns the given stress pattern and then shows a preference for the native stress pattern. How the stress pattern is important was shown in one study , where they tried to teach to kids associations between phonological dissimilar or similar nonsense words, and bilingual kids succeeded later than monolinguals.
To summarize, it seems that the language processing for bilingual infants and the way they process and acquire language is unique.
I tried to speak a bit, but it was really difficult with my parents. I learned to say the word “Cojeto” (“Co je to” is “What is it?” in Czech), but my father sometimes didn’t understand what I wanted. Then I better learned to point with my finger to things and make noise — it seemed to finally work (hurray, I found an international way of communication!). Anyway, even pointing can be quite frustrating with my parents. Sometimes they are so stubborn. Fortunately, training helps and they get better every day. I tried few more words without much success. Then I found “Nenene” (“no” in Czech), which is a lot of fun, and “brm brm” to tell them that I see a car or that I want to go on wheels and I was sooo happy when they finally both understood what I meant. When I am in a good mood, I say “ja” (“yes” in German). Luckily I can call them “mama” and “tata” and then they come smiling to me and hug me. Or we just play a “baf” game and I fall on my bottom from how much I have to laugh. Then I forget my frustration from not being able to tell what I want.
It might seem that the bilinguals start to speak later and their vocabulary is smaller, but this feeling is not based on any real studies. When summing up the words from both languages, it seems that the vocabulary of both bilingual/trilingual and monolingual children has the same size . It seems smaller only because they mix the words (code mixing) from both languages. Why they do so? Mixing words makes total sense – they have only restricted capacity for new words and they do not have a capacity to learn each word in both languages [11, 11b]. Later, when their vocabulary grows, they easily separate these languages .
There is one really funny thing about my parents. When we go around a dog, a pig, or a cow, they try to make the sound of the animal. But their animals always make different sounds. Mum says “haf haf”, “chro chro”, or “bú bú” and daddy says “wuff wuff”, “oink oink“ and “muh muh”. The funniest thing is when they see a sheep and daddy says to me: “Mäh mäh”. Then my mum starts complaining: “This is not a goat which makes ‘mé mé’, this is a sheep and it makes ‘bé bé’.” Isn’t it funny? To me every sheep sounds different, some say more “mäh mäh” and some say rather “bé bé”. But maybe when I will grow up, I will finally hear them properly and then I can tell my parents which of them is more right.
Raising bilingual children can be done in several ways and it is not clear which one is more effective or better . You can use one-person-one-language strategy (OPOL, each of the parents speaks consistently with one language), Minority language at home (MLAH, speaking one language at home and the one which dominates in country, in all the other occasions), or Time and place approach (T & P, speaking the second language in given times and situations). The selection of the strategy might depend on if you are fluent in all the languages and which language is the majority language in the country. Anyway it is important to note, that it was shown that the kid is not getting confused even when the same parent is switching between different languages as there are many cues that help the kid to distinguish which language is currently spoken, so do not be worried too much that you might confuse your kid. What is important is to expose the kid enough to the foreign language.
Some people around say that I will have a big advantage from being multilingual. They speak a language which I do not understand. They say that my brain is developing in a special way to maintain multiple languages and therefore it will gain a higher structural and functional plasticity also for other tasks, improving my learning and memory skills. That because my brain has to manage to switch between multiple languages, it develops enhanced executive control abilities. I am not sure what it should mean, but I guess it is good. I also heard some other things which I really did not understand – that recovery of my linguistic skills after a stroke might be quicker and that I might get a later onset of Alzheimer. Hmm, but how is all of this useful to me when I am not able to talk? When I have no way to tell my parents what I want? I would give anything to be able to speak to them. But which word should I select? Czech, English, German? I always think about starting to speak but then immediately I am voiceless. My mum cuddles me and tells me that I should not worry about it, that starting to speak a bit later is normal and that all the advantages will overgrow this small negative. I want to trust her. But anyway, I still would like to speak right now! At least I can understand what my parents are telling me. I also question myself, if kids on a playground will understand me. I understand them. But will they understand me?
Recent studies have shown that bilingual children have improved efficiency of cognitive processing and are better protected from cognitive decline compared to monolinguals. It seems that the necessity to manage multiple languages train the kids’ brain from the early beginning and leads to permanent changes of the brain of these kids. The bilingualism seems to contribute to neuroplasticity, which protects them from cognitive decline (e.g., the onset of dementia is on average 4 years later for bilinguals than monolinguals , delayed onset of Alzheimer disease [16, 16b, 16c], etc.).
What exactly happens in the brain of bilinguals? Recent studies using neuroimaging methods reveal that there are several modifications of their brains, enabling them to rely on more efficient processes for cognitive tasks. In the brain of bilinguals, both languages are jointly activated  and compete for selection, possessing big challenges and training for the brain. Bilinguals show greater gray matter volume especially in perceptual/motor regions . In parallel there is evidence that they also have greater white matter (WM) integrity  which might explain why bilinguals outperform monolinguals in the tasks when the quick response time is needed. Furthermore, they have stronger functional connectivity between brain regions, enabling more efficient distribution of the work within the brain. There was also measured less frontal activation during execution of nonverbal tasks, suggesting that managing mental tasks costs them less energy and that they are able to better distribute the work within the brain and work more effectively. [20, 21]
And what about trilingualism? That is not that easy to say. In general, we might expect that when a person is repeatedly faced with a challenging task, it should exhibit improvements in a cognitive processes. This was observed in many cases starting with action video games to playing a musical instrument. Nevertheless, if the cognitive demand is too high, it might not lead to a better performance. In the same way, we might see a bilingualism and trilingualism. There is various evidence on if trilingualism actually has benefits over bilingualism or not . It seems, that in some cases, it might have even higher benefits than bilingualism (such as a greater cognitive reserve in adults, [23, 24].), sometimes the benefits are the same (such as in inhibitory control [25, 26]), and in some cases, it even fails to show the benefits observed for bilingual kids (e.g. toddlers and infants do not show the same benefits in memory generalization tasks as bilingual babies [27, 28].).
To summarize, it seems quite clear that the benefits of multilingualism definitely outweigh the negatives. Learning multiple languages in childhood causes several functional and structural changes in their brain which delay the cognitive decay and improve the executive cognitive processes. The benefits are the strongest for the early learners but it is never too late to train your brain :-).
Pharo, Lars Kirkhusmo. “Multilingualism and Lingua Francae of Indigenous Civilizations of America.” Multilingualism, Lingua Franca and Lingua Sacra (2015): 467.
Werker, Janet F., and Krista Byers-Heinlein. “Bilingualism in infancy: First steps in perception and comprehension.” Trends in cognitive sciences 12.4 (2008): 144-151.
Peal, Elizabeth, and Wallace E. Lambert. “The relation of bilingualism to intelligence.” Psychological Monographs: general and applied 76.27 (1962): 1.
The roots of bilingualism in newborns. Byers-Heinlein K, Burns TC, Werker JF Psychol Sci. 2010 Mar; 21(3):343-8.
Visual language discrimination in infancy.Weikum WM, Vouloumanos A, Navarra J, Soto-Faraco S, Sebastián-Gallés N, Werker JF Science. 2007 May 25; 316(5828):1159.
Bosch and Sebastian Gallen 2003 – bilingual infants Spanish vs. Catalan, Burns 2003 – Eng. vs French, Goto, 1971 – Japanesse, Werker, 1981 – Hindi vs. EngDistinguish phonological and lexical differences REF CHILDHOOD BILINGUAL
 Abercrombie 1965 Stress pattern REF FROM CHILDHOOD BILINGUAL
 Same size vocabulary biling monoling when summing up
 Byers-Heinlein, Krista, and Casey Lew-Williams. “Bilingualism in the early years: What the science says.” LEARNing landscapes 7.1 (2013): 95.
[11b]Lanza, Elizabeth. Language mixing in infant bilingualism: A sociolinguistic perspective. Oxford University Press, 1997.
 Genesee F, Boivin I, Nicoladis E. Talking with strangers: A study of children’s communicative competence. Applied Psycholinguistics. 1996;17(4):427–442. doi: 10.1017/S0142716400008183.
 Braun, Andreas, and Tony Cline. Language strategies for trilingual families. Multilingual matters, 2014.
 Bialystok E, Craik FIM, Luk G. Bilingualism: consequences for mind and brain. Trends Cogn Sci.2012; 16(4):240–250. DOI: 10.1016/j.tics.2012.03.00
Bialystok E, Craik FIM, Freedman M. Bilingualism as a protection against the onset of symptoms of dementia. Neuropsychologia. 2007; 45(2):459–464. DOI: 10.1016/j.neuropsychologia.2006.10.009
 Gold BT. Lifelong bilingualism and neural reserve against Alzheimer’s disease: A review of findings and potential mechanisms. Behav Brain Res. 2015; 281:9–15. DOI: 10.1016/j.bbr.2014.12.006
[16b]Guzmán-Vélez E, Tranel D. Does bilingualism contribute to cognitive reserve? Cognitive and neural perspectives. Neuropsychology. 2015; 29(1):139–150. DOI: 10.1037/neu0000105
[16c] Schweizer TA, Ware J, Fischer CE, Craik FIM, Bialystok E. Bilingualism as a contributor to cognitive reserve: Evidence from brain atrophy in Alzheimer’s disease. Cortex. 2012; 48(8):991–996. DOI: 10.1016/j.cortex.2011.04.009 [PubMed: 2159637
 jointly activated from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5446278/pdf/nihms853932.pdf
 Olsen RK, Pangelinan MM, Bogulski C, et al. The effect of lifelong bilingualism on regional grey and white matter volume. Brain Res. 2015; 1612:128–139. DOI: 10.1016/j.brainres.2015.02.0
 Voelker P, Piscopo D, Weible AP, et al. How changes in white matter might underlie improved reaction time due to practice. Cogn Neurosci. 2016 Apr.2016:1–7. DOI:10.1080/17588928.2016.117366
 Grundy, John G., John AE Anderson, and Ellen Bialystok. “Neural correlates of cognitive processing in monolinguals and bilinguals.” Annals of the New York Academy of Sciences 1396.1 (2017): 183.
 Paradis, Michel. The handbook of the neuroscience of multilingualism. John Wiley & Sons, 2019.
 Schroeder, Scott R., and Viorica Marian. “Cognitive consequences of trilingualism.” International Journal of Bilingualism 21.6 (2017): 754-773.
 Chertkow H, Whitehead V, Phillips N, Wolfson C, Atherton J, Bergman H. Multilingualism (but not always bilingualism) delays the onset of Alzheimer disease: Evidence from a bilingual community. Alzheimer Disease and Associated Disorders. 2010;24(2):118–125.
Perquin M, Vaillant M, Schuller AM, Pastore J, Dartigues JF, Lair ML, Diederich N. Lifelong exposure to multilingualism: New evidence to support cognitive reserve hypothesis. PLoS ONE. 2013;8(4).
 Poarch GJ, Bialystok E. Bilingualism as a model for multitasking. Developmental Review. 2015;35:113–124.
 Vega-Mendoza M, West H, Sorace A, Bak TH. The impact of late, non-balanced bilingualism on cognitive performance. Cognition. 2015;137:40–46.
 Brito NH, Grenell A, Barr R. Specificity of the bilingual advantage for memory: examining cued recall, generalization, and working memory in monolingual, bilingual, and trilingual toddlers. Frontiers in Psychology. 2014;5:1369.
 Brito NH, Sebastián-Gallés N, Barr R. Differences in language exposure and its effects on memory flexibility in monolingual, bilingual, and trilingual Infants. Bilingualism: Language and Cognition. 2014:1–13.
I am happy to announce that we got granted a new research project with the title “Multimodal representation of robotic actions and tasks applied to Learning by demonstration”, where I am a principal investigator. The project starts on 1.7.2021 and will last for 3 years. Looking forward for a great research and nice collaborations :).
Our paper “Embodied Reasoning for Discovering Object Properties via Manipulation” was accepted for ICRA 2021.
Abstract: In this paper we present an integrated system that includes a reasoning from visual and natural language inputs, action and motion planning, executing tasks by a robotic arm, manipulating objects and discovering their properties. The vision to action module recognises the scene with objects and their attributes and analyses enquiries formulated in natural language. It performs multi-modal reasoning and generates a sequence of simple actions that can be executed by the embodied agent. The scene model and action sequence are sent to the planning and execution module that generates motion plan with collision avoidance, simulates the actions as well as executes them by the embodied agent. We extensively use simulated data to train various components of the system which make it more robust to changes in the real environment thus generalise better. We focus on the tabletop scene with objects that can be grasped by our embodied agent, which is 7DoF manipulator with a two-finger gripper. We evaluate the agent on 60 representative queries repeated 3 times (e.g., ‘Check what is on the other side of the soda can’) concerning different objects and tasks in the scene. We perform experiments in simulated and real environment and report the success rate for various components of the system. Our system achieves up to 80.6% success rate on challenging scenes and queries. We also analyse and discuss the challenges that such intelligent embodied system faces.
Smiles, laughter, giggles — as soon as the baby’s face lights up into a guileless smile, even the grumpiest passer-by melts and has to smile. Smiles are the reward for parents for all those sleepless nights and the manifestation of the connection which we yearn for from the first day of childbirth.
A smile tells us that everything is fine.
A smile makes us laugh back.
The smile makes us feel good.
When can a baby really start laughing? And why are they smiling?
What happens in the brain of a child and an adult when they laugh?
Does it have any effect on the development of the baby in the uterus and on the newborn, whether his mother often laughs or is depressed?
How can robotics help us to understand the effects smiles have on us?
Right after the delivery we take our babies and pack them into a cloth that holds their hands and legs close to their bodies, to let them feel hugged and safe. And so we tend to forget that they can already move, that back there in the womb, they were moving, touching, listening, observing. We greet every move of their hands and legs with huge applause. We observe how they start pseudorandomly move their hands around, kicking legs when they want to reach a toy as they are exploring options of their bodies. Like every mum, also I can spend many hours just watching how is the little baby exploring the world around. Slowly, as they touch by their hands and legs surrounding objects or themselves, they start to move more and more intentionally, and we focus on every step of their development with incredible attention. All this seems to be a continuous development with the starting point at delivery. We tend to forget about their first kicks in our belly, by which they were letting us know that they are already there, about those long nine months which they spent exploring the womb. How it was in there? How often they move, how important is their motor development inside the belly for their future motoric development at all?
Compared to other infrahuman primates (this means chimpanzees, apes, and others) human babies have delayed development [1, 2]. This means that for many weeks after the birth, the baby actually does not change much its behavior compared to what it was doing inside the womb. It is still not able to care for itself and needs long term support from the mother. It seems that there is a central movement pattern generator  in the brain, which creates various movements (either general movements of the whole body or isolated movements of the individual limbs) and this same pattern generator is serving the body both before and after birth . This gives us an amazing option to see, how was our precious bundle behaving inside our belly – which movements it used to do, how it sometimes accidentally touched its head by its hand, how it was sucking the amniotic water, how it was breathing, yawning, swallowing, kicking,… It takes up to 3 months after birth when these mainly random predefined motor patterns change to more intentional goal-directed movements, that are guided by the sensory system (which was also not fully developed in the first weeks after the delivery so the baby had to wait a bit longer to be able to make use of it) – and the baby can finally reach for a toy by hand or rotate in its direction…
Development of the motoric system
The motoric system is one of the first systems in the fetus’s body which starts to develop. It was shown that it is crucial for brain development that the motoric action provides an appropriate change in the sensorial world (e.g. we can feel that we touched something or we can see that something moved, etc.) (see e.g. Held and Hein  who performed a classic experiment with kittens). Thanks to modern ultrasound devices we can observe gradual changes in human fetus development in detail. First longitudinal studies were performed in the 1980s by Prechtl  and uncovered till that point many unknown phenomena about that secret world “inside”. From the studies of the motor patterns, we can nowadays observe also the brain connectivity of the fetus and see for example that good connectivity of some regions (prefrontal cortex, motoric network, supply motor regions, and temporal lobes) in the fetus brain is very important for later post-term motor development of the kids (Thomason, 2018). In this study, they found out that the connectivity also differs between female and male fetuses and discusses that this might be related to the later sex differences between infants. There are studies that show that female infants are better in fine motor skill development, while males in gross motor  or those female infants show more coordinated arm movements .
When do individual movements start?
As soon as in 8 weeks of pregnancy the fetus starts to move sideward its head . These first body movements are followed by generalized movements of the whole body in 9-10 weeks – the whole body is either moving slowly in a sequence (general movements) or all limbs, trunk, and neck are moved by quick phasic movements (startles) . It takes another week or two when we can (between 10-11 weeks) observe isolated local movements of one arm or leg. Isolated movements of some parts of the body are way more difficult to produce than whole-body movements both for the fetus as for the newborn. Teaching individual brain regions of the motor cortex to specialize for moving individual body parts will be under development for many years so it is no wonder that it is easier to generate in the fetus brain activity responsible for the whole-body movement. Just imagine when someone asks you to move one of your toes on your feet. Can you manage?
In the 10 weeks of pregnancy come hiccups. These movements are caused by repetitive short contractions of the diaphragm, which might last for several minutes and as you well know, they can be pretty annoying. Sometimes they might be so strong that they move the whole fetus in the womb.
If you wonder when the baby can already move its head to sign “no” or “yes” to your question by moving head back and forth or rotating it from side to side, it is actually as soon as in 11 weeks of pregnancy. At that time, the head becomes mobile and can move forward, backward, and rotate. It means that now we have to just find the way how to let the baby know that it can use it for communication with us and by observing the womb via ultrasonic devices, we can see if it likes or dislikes what we do 😊. What is great is, that this general pattern generator for limbs and head movements might create at this point accidental hand-eye contacts. These are totally not intentional, but still, provide important information for the fetus. These are the first self-contacts that enable later self-awareness of the baby and a better understanding of its own body. Same as we tried in our research where the robot was trying to touch itself to find out the length of its arms and torso as well as the position of its eyes (cameras) (Stepanova et al. , https://www.youtube.com/watch?v=zP3c7Eq8yVk&feature=emb_logo ).
In 11-12 weeks we can observe periodical breathing of the fetus and in 12 weeks we see stretches and yawns. It might impress you to observe your newborn yawning as it seems so mature. The fact is, that these movements which appear as soon as in 12 weeks of pregnancy will keep the same form and pattern for the whole life. Cool, right?
Sucking. You might be impressed by the newborn sucking reflex and then the strength of the sucking itself (at least some of the kids, some others need a lot of stimulation and their sucking is very weak). How quickly it finds the place to suck and how much it hurts when the baby actually sucks accidentally in the wrong place. No wonder, it has good training also for a few months before it comes to the world. After 12 weeks of pregnancy, the baby starts to drink the amniotic fluid, that is connected to first sucking and swallowing movements. As the sucking might happen in the womb all the time, after delivery, the baby will have to learn to connect this sucking behavior to feeding occasion – yet some of the kids would love to continue in the “sucking all the time” behavior 😊. Sucking reflex itself (meaning that the baby starts sucking as it gets nipple or something else to the mouth) develops around 32 weeks of pregnancy. (https://www.healthline.com/health/parenting/sucking-reflex#test).
At around 20 weeks, the baby starts to prepare for the outside world and first slow eye movements (20 weeks) followed by rapid eye movements (22 weeks) develop. 
As the fetus is growing and space is getting restricted, most arm movements end up not far from the face, and the amount of hand-face contact increases . The first movements are mainly unintentional, later there are also more and more non-reflexive responses to sensorial inputs – like to sounds, light (response to disturbances  or movements towards the source  (see also my earlier blog post about the language development), response to the maternal touch of the abdomen by an increase of arm, head and mouth movements  as well as changes in fetal heart rate. But still, most of the movements are only ‘‘motor babbling’’ – the spontaneous activity of the fetus.
By about week 21, the fetus begins to develop a regular schedule of movement  The startle reflex is present in half of all fetuses by week 24 and in all fetuses by week 28.
After the birth comes with sensory inputs also new options and new movements. The baby must deal with the gravity, it has to learn how to deal with the lung ventilation and so we can observe sneezing and coughing and so on.
The motor cortex of the fetus
It is believed  that the fetus is creating connections between motoneurons and muscles via these spontaneous movements when via general and isolated movements it receives various types of stimulations. In this way, specialized areas in the brain that are created to operate given body parts might be created. When individual body parts touches another body part or the outside world as well as when an outside stimulus triggers some sensation, all of these contributes in the creation of new neural connections and enable the fetus later to repeat any movement. You can see research about creating these somatosensorial maps in the brain of the humanoid robot (https://www.youtube.com/watch?v=ONiKEtIJc3Q&feature=emb_logo , Hoffmann ).
How often do babies change their position in the womb? And should you be worried if they do not kick enough?
I guess that every mother goes at some point through
that scary moment. Sitting on the sofa and thinking that the baby did not move
for too long. Starting to worry, counting seconds, getting scared if everything
is ok with the baby. Should I call a doctor? Should I worry? What is wrong with
this baby? Or when you come to the monitor before delivery and you everybody
around gets “a nice” monitor immediately, only you are the one staying there
longer, breathing deeply, sisters changing your position that the baby finally
moves…It is easy to get stressed.
Up to 16 weeks of pregnancy the frequency of movements gradually increases and around 20 weeks, calm periods (without any type of movements like general or isolated movement, hiccups, mouth movements, etc.) are very short (maximum 13 minutes, de Vries et al., 1985). One of the cool things about all those types of movements mentioned above is that our baby can change its position in utero. What is interesting, that this change in position happens way more frequently in the first half of the pregnancy (up to 25 changes per hour) than in the second half. So, when we can finally feel the first baby kicks (at about 20 weeks of the pregnancy), changes in position are way rarer . It is not that surprising if you consider that the baby does not have in the womb that much space anymore. It is believed, that this decrease in frequency is also connected to the emergence of inhibitory cortical influences . Second or third children may have more stretching room in the womb than first babies because a woman’s uterus is bigger and the umbilical cord longer after her first pregnancy. These children usually get more motor experience in utero and tend to be more active infants . Keep in mind that there are huge differences in movements and their strength. There are studies, which found out that boys might be kicking more than girls  with higher average movements at 20, 34, and 37 weeks. Anyway, the study was done only on 37 babies, which is really not a relevant sample (https://www.livescience.com/62928-why-babies-kick.html ). When we go for a monitor, you can see many mothers discussing if they should eat or not before it to make the baby move. What was found out is, that most of the movements of the fetus are not connected to the amount of glucose in the mother’s blood (it does not matter if she is after or before meal)? The only exception is breathing movements of the fetus which are easier observable after the meal .
So you do not have to be worried if your baby kicking patterns are changing if you feel its kicks some weeks later or earlier if sometimes it is kicking more and sometimes less, especially if you are distracted by your own activity. Anyway, by week 29, you should be feeling your baby moves well. On average, you should feel some type of their movements (kicking, rolling, stretching, even light movements count) at least 10 times within 2 hours . If you do not get these 10 kicks within 2 hours while being in a calm environment resting, you should change position, wait an hour or two, calm down and try to count once again. If you do not count a reasonable amount of movements even then, then it is a proper time to call your pediatrician, to make sure that everything is ok.
There is much more to tell about the motor development of the baby in the womb, especially about its brain development, but let’s continue some other time. There are still many secrets even when we have nowadays very good tools to look inside the womb. What we know for sure is that the movements of the baby in the womb (and all the sensorial inputs it gets thanks to them) are very important for its future development.
 Hadders-Algra, Mijna. “Early human motor development: From variation to the ability to vary and adapt.” Neuroscience & Biobehavioral Reviews 90 (2018): 411-427.
 Einspieler, Christa, Peter B. Marschik, and Heinz FR Prechtl. “Human motor behavior: Prenatal origin and early postnatal development.” Zeitschrift für Psychologie/Journal of Psychology 216.3 (2008): 147.
 PRECHTL, HF R. “Continuity and change in early neural development.” Clinics in developmental medicine 94 (1984): 1-15.
 Held, Richard, and Alan Hein. “Movement-produced stimulation in the development of visually guided behavior.” Journal of comparative and physiological psychology 56.5 (1963): 872.
 Prechtl, Heinz F., ed. Continuity of neural functions from prenatal to postnatal life. No. 94. Cambridge University Press, 1991.
 Touwen, Bert CL. Neurological development in infancy. London: Heinemann, 1976.
 Piek, Jan P., et al. “Limb and gender differences in the development of coordination in early infancy.” Human movement science 21.5-6 (2002): 621-639.
 Lüchinger, Annemarie B., et al. “Fetal onset of general movements.” Pediatric research 63.2 (2008): 191-195.
 De Vries, Johanna IP, G. H. A. Visser, and Heinz FR Prechtl. “The emergence of fetal behaviour. II. Quantitative aspects.” Early human development 12.2 (1985): 99-120.
 Stepanova, Karla, Tomas Pajdla, and Matej Hoffmann. “Robot Self-Calibration Using Multiple Kinematic Chains—A Simulation Study on the iCub Humanoid Robot.” IEEE Robotics and Automation Letters 4.2 (2019): 1900-1907.
 Birnholz, Jason C. “The development of human fetal eye movement patterns.” Science 213.4508 (1981): 679-681.
 Fagard, Jaqueline, et al. “Fetal origin of sensorimotor behavior.” Frontiers in neurorobotics 12 (2018): 23.
 Valman, H. B., and J. F. Pearson. “What the fetus feels.” British medical journal 280.6209 (1980): 233.
 Lecanuet, Jean-Pierre, and Carolyn Granier-Deferre. “Speech stimuli in the fetal environment.” Developmental neurocognition: Speech and face processing in the first year of life. Springer, Dordrecht, 1993. 237-248.
 Marx, Viola, and Emese Nagy. “Fetal behavioural responses to maternal voice and touch.” PloS one 10.6 (2015): e0129118.
 Vaughan, Christopher. How Life Begins. Dell Publishing Company, 1997.
 Hoffmann, Matej, et al. “Robotic homunculus: Learning of artificial skin representation in a humanoid robot motivated by primary somatosensory cortex.” IEEE Transactions on Cognitive and Developmental Systems 10.2 (2017): 163-176.
 Almli, C. Robert, Robert H. Ball, and Mark E. Wheeler. “Human fetal and neonatal movement patterns: Gender differences and fetal‐to‐neonatal continuity.” Developmental Psychobiology: The Journal of the International Society for Developmental Psychobiology 38.4 (2001): 252-273.
Have you ever thought about what a baby is actually seeing when it is still in the belly? Or you thought, as well as my daughter, that since the baby is in the dark and in the water, it does not open eyes until it gets out and development of vision just starts at the second when the baby leaves the belly? Yet it seems, that not only that visual system as the whole is developing already since the 4th week of pregnancy (wiring important neural connections, building up tissues, lenses, muscles, and all the other necessary stuff,…), but the baby can actually see inside the womb some light, after 26 weeks it can open eyelids and it keeps eyes open when awake, it reacts to intense light and some recent studies suggest even so crazy ideas such as that babies might recognize faces already in the womb – as it seems that the baby reacts to the light in the shape of face more than to light in different shapes – crazy, right?
Development of the visual system
From the beginning of the 2nd month of pregnancy (at around 4 weeks), the development of the visual system and eyes starts and for next few weeks you can see such a rapid development of all the important parts of the visual system that it seems almost incredible. The fetus is really busy with creating all eye structures and corresponding neural connections. As stated in very nice course : “Eye development during the first trimestr of pregnancy is like watching the grand finale of Fourth of July fireworks. Cells and tissues develop quickly and simultaneously to form various eye structures.”
At 4 weeks precursor to 2 optic nerves (optic stalk) is created (it is developed at 36 days). Optic nerves create a crucial connection between eyes and forebrain (each one connecting eyes to the one side of the brain) [2, 4]. In parallel, at 4 weeks, first cells start the baseline for lens (which will help the kid to focus on objects), at 32nd day we can already identify them and by the end of the 2nd month of pregnancy they will grow to the size which they will have at birth . Also, at 30-35 days of pregnancy iris (which helps to reduce the amount of light coming to the eye) start to develop and 2 weeks later it is ready. At 4 weeks of pregnancy also extraocular muscles are created, as well as the cornea is developing at this time. The retina (layer in the back of the eye covered by photoreceptors which perceives and processes light and sends signal to the brain) starts its development around 8 weeks of the gestation age.
Eyelids start to form around 8 weeks. They are closed and serve as a protection for other developing eye structures . At 16 weeks, the eye is starting to pick up on light and can perform slight movements from side to side in response to light . By this time, eyelids are still fused and closed. From the beginning of the 5th month till the end of the 6th month, eyelids are slowly separated. At 26 weeks (7 month) of gestation age, highly-hydrated cornea starts to be transparent and enables eyes to sense light. Eyelids finally open.  Since then the baby keeps eyelids open when awake. It seems, that the baby is from now on able to sense the movement of bright light outside your body – if you flash bright light to the belly, you might get “kick” response. Between 28 – 30 weeks, rapid eye movements and sleep patterns appear. As will be mentioned later, the REM sleep and sleep patterns are crucial for healthy vision development as they synchronize brain waves of visual system with retina .
How dark is it in the womb and how the baby reacts to the light?
As we could see, in the last 2 months of the pregnancy (from the week 30-32), the visual system is already developed enough to be able to react to the incoming light and process some incoming visual information – although with a lower acuity (fetuses would only be able to see large objects or smaller objects close by) and mainly in the grayscale (as the cones are not yet developed). So the question is – how much the baby can actually see in there? Marco del Giudice used measured transmission coefficients of biological tissues and modeled light transmission from the external environment to the uterine cavity. He found out that for the naked woman, around 0.1-1 % of ambient light might get inside the womb. This amount is highly variable and depends on the thickness of the tissue (how fat is the belly) and the strength of the external illumination [7, 8].
Now the question is, how much light is needed for prenatal visual experience? Glass  review of studies with newborns (term and premature) suggest that the amount of light needed for fetal vision is way lower than the one of mature individuals.  makes a conservative estimate of the amount of light permitting vision of fetus to 10 lx (this is light which an adult needs to read a text). Study published by  shows that we can expect that fetus in the womb receives at least 10-100 lx, therefore its visual experience is enabled.
In the recent study (2017)  V. Reid et al. used a 4D scanner to explore how fetus reacts to the light source in different shapes. On top, the response of the fetus was recorded when the light source was moving. The visual stimuli was projected through the uterine wall and fetal head turns were examined. They found out that fetus in the third trimester looks toward three dots which are configured upright like a face significantly more than to three inverted configuration dots (the stimuli they used is shown on the figure – and as you can see, interpreting it as face is somehow not that intuitive…at least not for me 🙂 ).
So it seems that the womb is not that dark place after all…
What can we do during pregnancy for the vision of our baby?
(…how we can help to maintain healthy development of eyes and vision of our precious one?)
It seems that the most important for visual system development is genetic coding  (you cannot do much about this part :)), certain nutrients – particularly vitamin A [1,2,5](you can find it in fruits, dairy products and mainly in leafy vegetables), and keeping REM sleep and sleep cycles of the baby undisturbed. Why is REM sleep and sleep cycles so important for the visual system development? During its life in the womb, the brain of the fetus creates topographic relationships between retina, lateral geniculate nucleus and primary visual cortex. To create those mappings correctly, REM sleep (accompanied by rapid eyes movement) and sleep cycles seem to be crucial and sleep disturbance or deprivation might result in the fact that these important bindings are not created properly .
So maybe we should not try too often to make the baby move by pushing, flashing light to the belly, or impose it to loud noises (you shouldn’t anyway)…
Is it good to stimulate a fetus by light before birth or not?
There is no doubt that simulation of the vision after birth is very important for newborn visual system development . On the other hand, there is contradictory evidence whether simulation of visual system before birth might be beneficial or not.  states that it is not important and the visual system cannot benefit from it. On the contrary, there is a study done on pregnant mice , which were kept in complete darkness and after birth, their babies were more likely to have vision problems. This study states that fetal mice require light exposure in utero during early gestation for normal vascular development in the eye. In parallel, there were done studies on the pregnant women who live at northern latitudes with pregnancy lasting during the darkest months of the year. Surprisingly, their babies have actually increased risk of certain eye disorders [2,6]. These studies suggest that stimulation of the visual system is important – at least to some extent. Anyway, it doesn’t mean that you should be flashing light to your belly to keep your baby stimulated. Just do not stay inside a cave all the time :).
Note: Keep in mind that pupils start to respond to light earliest in the 8th month and it takes another 2 weeks until the response is consistent. That is why premature babies (born before 34 weeks of gestation) need eye protection to let the eye finish its full development. The need to protect the underdeveloped visual system from light outweighs the benefits of its stimulation until the premature infant is reaching the age typicall for leaving the womb – 40 weeks of gestation. 
Can a baby in the womb cry?
Lacrimal glands (where tears are produced) begin to form already during the 6th week of pregnancy. However, they do not produce no tears until the third month after birth, which is why infants shed no tears when they cry. 
Does a fetus/newborn see in colors?
It is important to notice that the retina development goes on from 8 weeks of gestation age till 3-4 months after birth. So at the time when the baby first comes to the world, the optic nerve is still not fully myelinated and rods (photo receptors responsible for grey-scale vision enabling us to see in the dark) are way better developed than cones (enabling color vision). Therefore, newborn cannot see in colors, but mainly perceives shades of gray and discriminates between dark and light. The color vision (enabled by red, green and blue cones) will come later – around 3 months of age – together with maturation of cones . Maturation of the whole visual system and corresponding neural areas will still go on…
 See, Angela Wai-Man, and Margaret Clagett-Dame. “The temporal requirement for vitamin A in the developing eye: mechanism of action in optic fissure closure and new roles for the vitamin in regulating cell proliferation and adhesion in the embryonic retina.” Developmental biology 325.1 (2009): 94-105. https://www.ncbi.nlm.nih.gov/pubmed/18955041
Our paper “Simultaneous task and motion scheduling for complex tasks executed by multiple robots” was accepted for ICRA 2020.
Abstract: Coordination of multiple robots operating simultaneously in the same workspace requires the integration of task scheduling and motion planning. We focus on tasks in which the robot’s actions are not necessarily confined to small volumes, but can also occupy a large time-varying portion of the workspace, such as in welding along a line or drilling a hole. Optimization of such tasks presents a considerable challenge mainly due to the fact that different variants of action execution exist, for instance, there can be multiple starting points of lines or closed curves, different filling patterns of areas, etc. We propose a generic and computationally efficient optimization method which is based on constraint programming. It takes into account the kinematics of the robot and guarantees that the motion trajectories of the robots are collision-free while minimizing the overall makespan. We evaluate our approach on several tasks of varying complexity: cutting, additive manufacturing, spot welding, inserting and tightening bolts, performed by a dual-arm robot. In terms of the makespan, the result is superior to task execution by one robot arm as well as by two arms not working simultaneously.