Byte Tank - Pedro Lopes Blog https://lopespm.com/ Bluetooth Headphones Safety https://lopespm.com/notes/2025/02/22/wireless-headphones.html https://lopespm.com/notes/2025/02/22/wireless-headphones.html After nearly 4 years of continuous usage at work, taking calls and listening to audiobooks during long walks, my wired in-ear Bose QuietComfort 20 MK2 Active Noise Cancelling Earphones are finally starting to become faulty1, so I went on a search for a replacement.

Searching for replacement wired ANC earbuds

To my surprise, not only have these types of earphones been discontinued by Bose, but the selection of wired in-ear active noise cancelling headphones is incredibly slim. After an extensive search, I’ve found Asus ROG Cetra II and Bang Olufsen B4, which appear to offer inferior quality in terms of Active Noise Cancelling (ANC), when compared to recent highly rated Bose, Sony or Apple headphones, which are either wireless or are over-ear / on-ear. Quality in-ear2 ANC earbuds are almost exclusively wireless.

Health

I understand the popularity of wireless earbuds. They are extremely practical and neat. It is not surprising that penetration of bluetooth headphones has jumped considerably in recent years, but given that I use my headphones for several hours each day, it is not an enticing proposition to have two wireless devices extremely close to my brain, even if they output a low amount of EMF radiation. The (cumulative) dose makes the poison.

Several experts 3 4 5 claim there is no danger in prolonged usage of these devices, because they don’t emit ionizing radiation, but it has been reported both in studies and anecdotally that exposure to high levels of EMF radiation might have detrimental effects long term.

I’ve found it challenging to uncover quality studies that correlate the usage of bluetooth headphones and health effects. I assume this correlation is hard to experiment upon with statistically significant results, due to how long this exposure needs to happen, and other confounding environmental and behavioural factors. Regardless, one of these studies draws a significant link between the usage of bluetooth headset and thyroid nodules.

Unknowns Risks

Widespread adoption of these devices is still relatively recent, and we should be humble enough to acknowledge that we don’t have the full picture clearly laid out of the all repercussions related to their long term usage, especially when in close proximity to one of the most valuable organs of our body, our brain.

It’s not what is known that concerns me, but rather what is not known. Until proven the contrary, I will continue to play it safe, and avoid using wireless earbuds.

Hopefully a larger swath of the population will increasingly exercise caution and awareness on this issue, signalling manufacturers to drive the supply of competitive good quality wired in-ear ANC headphones that empower the consumer to make the best choice for their use case, and their health.

  1. The battery is still pristine, but the left earbud started to give out a random noise when active noise cancelling is enabled.↩

  2. I’m exclusively looking for in-ear headphones because of how challenging it is to find over-ear or on-ear headphones that doesn’t cause discomfort after several hours, when using glasses. I also appreciate the style of in-ears, although that is secondary.↩

  3. Effect of Bluetooth headset and mobile phone electromagnetic fields on the human auditory nerve - Marco Mandalà, Vittorio Colletti, Luca Sacchetto, Paolo Manganotti, Stefano Ramat, Alessandro Marcocci, Liliana Colletti↩

  4. Are Bluetooth Headphones Safe? - Dr. Matt MacDougall & Dr. Andrew Huberman↩

  5. How Unhealthy Are Your AirPods? - Doctor Mike↩

]]> 2025-02-22T00:00:00+00:00 The Story of the Human Body Book: Lessons https://lopespm.com/notes/2025/02/10/story-human-body-lessons.html https://lopespm.com/notes/2025/02/10/story-human-body-lessons.html I’ve recently finished reading The Story of the Human Body: Evolution, Health, and Disease, a book from Daniel Lieberman of how the human body evolved over millions of years, that I would recommend all Humans to read. These are are my main takeaways:

Why we are, the way we are

It is all about how energy is used

Us humans and other living beings are essentially organisms that use energy to reproduce and maintain ourselves.

  • Natural selection prunes off organisms that don’t energy in a way that is suited to their conditions and environment
  • Every energy transformation feature / trait has a cost. For example: walking upright allows one to walk using less energy, but lacks the speed of a quadruped.

Long Distance Walking / Running Machines

What drove selection for bipedalism

Hominid bipedalism was likely to have been initially selected to help the first hominids forage and obtain food more effectively in the face of major climate change

Between 10 and 5 million years ago, the earth climate cooled considerably, and overall effect in Africa that caused rainforests to shrink and woodlands to expand. If you were to be in the heart of the rainforest, you likely wouldn’t have noticed much of a difference, but if you were in the margins, this change must have been stressful. As the forest shrinks and becomes woodland, ripe fruits become less abundant, more dispersed and more seasonal. These changes would require you to travel farther to get the same amount of food.

Walking on two legs is more energy efficient for longer routes. Main trade off is less speed, because we could not gallop. We are 3x more efficient moving a distance of 6km than a chimp.

The consequences of bipedalism

Not only was endurance walking useful, but also long distance running was likely selected for scavenging. Running to fresh carcasses, running with food and, persistence hunting, where a big animal is followed by running and walking on sun. They need to find shades and pant, and eventually collapse due to heat stroke.

  • We are unique in that we have sweat glands all over the body. Other apes mostly have on the palms of their hands. And we. Don’t have fur, allowing the air to come in contact with sweat and cool your body down. This is essential for long distance running and walking.
    • Other animals have fur that reflects solar radiation, but does it allow body to cool down through sweat

Other features that help us to run effectively. Some of them not specially helpful for walking, like shorter toes:

  • Big gluteus maximus, mostly active when running.
  • Nuchal ligament is a back neck ligament that connects head to arm that helps stabilize the head while running. It was developed independently in humans and other animals well adapted for running.
  • Big semi circular inner ear canals to have more signals to allow for stabilization of our inner systems, such eye actions to compensate for the jiggle.
  • Predominance of slow twitch fibers in legs. More endurance, thus compromising speed.
  • Narrow waists, wide shoulders, shorter toes
  • Long noses are selected for walking long distances without drying up. Nose humidifies air coming into the lungs, which is necessary. And retains moisture going out. Trade-off is that lungs have to work a bit harder

All of these walking and running adaptations made us clumsy climbing trees, like shorter toes, but also freed up our hands to interact with the world, create the first tools and throw objects (we are especially good at throwing, having shoulders and upper body that makes it ideal for it), which not only provided some protection, but also allowed us to obtain and process fallback / lower quality foods (which were essential due to scarcity from global cooling climate).

Only mad dogs and Englishmen go out in the midday sun ― Noël Coward

Better dexterity, tools, cognition, less digestive cost, more energy availability

In the average adult human, the brain represents about 2% of the body weight, but accounts to about 20% of total energy consumption.

As the ability to create and use tools allowed the gathering and processing of fallback foods, more usable energy was able to be extracted. Processing foods to smaller chunks, or cooking it, largely increases how much energy can be obtained during digestion, but also requires less intense chewing (we didn’t need as bigger teeth as australopithecus, which resulted in use losing the snout due to less space being needed in the head) and shorter intestines, which reduce energy required for digestion, increases surplus energy to be used by the brain. These factors selected hominids with better manual dexterity and better cognition.

Our manual dexterity, and what it unlocks, plays a big role in our evolution.

Energy hungry brain, and relation with fat

If our brain1 is deprived of glucose for even 1 minute or two, it causes irreparable damage. This likely selected humans to become unusually fat. 400 to 500 calories daily, are for the brain alone. Extra intense thought only increases hourly usage by 5 calories per hour.

Hunter gatherer mothers have to expend about 2.3k calories for themselves alone to raise kids. Babies are unusually fat. 60 percent of their energy consumption is for the brain. It takes 12M calories to grow a child into a full grown adult. Twice as much as those needed for a chimpanzee.

Our way of living affected the size of our gut so profoundly, that now we are pretty much dependent on cooking to get the caloric intake we need for our body and brain.

Exercise is not great to lose fat (unless you don’t succumb to overeating after exercise) but it’s good to prevent fat gain. It increases muscle (and not fat) sensitivity to insulin and makes the muscles more readily available to store energy, and also increases the number of mitochondria that burn fat and sugar.

  • A lean man that doesn’t exercise has twice the risk of dying, compared to obese men that engaged in regular physical activity

High offspring investment

Life is fundamentally a way of using energy to make more life.

  • If the environment is risky: favors quick returns, where little investment is made during life, but a lot during reproduction. Such as spiders, salmon that lay a lot of eggs, hoping that a lucky few will survive
  • But when resources are predictable and infant mortality is low, the strategy of humans, elephants and apes is feasible. maturing slowly and investing a lot in raising the few offspring

Cost of clear speech: choking

What good is it to have a good idea, if you can’t communicate it? One of the main features we developed as humans was our ability to use speech to clearly communicate thoughts, ideas, plans, and other information.

The price is that we are the only mammal that can risk asphyxiation when swallowing something too large or imprecisely2, because of the big common space behind the tongue through which food and air both travel to get either into the esophagus or trachea, a consequence of our short and retracted faces and selection of anatomy that favors clear speech. As a result, food sometimes gets lodged on the back of the throat, blocking the airway.

When you are having lunch with friends, consider that you are doing two things: speaking clearly, and swallowing a little dangerously.

Diseases

We’ve created a system that makes people sick through a surplus of energy, and keeps them alive without needing to turn down the energy flow

Lieberman presents the concepts of Mismatch diseases and Dysevolution:

  • Mismatch diseases: Diseases that occur because our bodies are poorly or inadequately adapted to environments in which we now live. An example would be eating large amounts of sugar or being very physically inactive leads to problems like diabetes or heart disease that then make us sick
  • Dysevolution: Form of cultural evolution where the symptoms of a mismatch disease are treated, but not the cause. One example of this is cavities, which are treated with dentists, and not by removing the sugar from our diet, which is the underlying cause. Lieberman emphasizes that mismatched diseases and their causes must be understood fully to treat the true cause.

Too much energy surplus

Our ancestors ate fruits as sweet as carrots, but as energy dense foods had their availability increased3, more humans moved from subsistence farming and physically strenuous jobs to lower physical effort jobs (in the USA, only 11 per cent are actually factory workers. The rest are in management, services, wealth management and banking services), the same mechanisms that made us effective is storing energy and craving certain foods, are also the ones that have become maladapted to our current environment.

It has been only about 300 generations since we’ve been subsistence farmers, which didn’t give enough time for natural selection and body adaptations to develop.

Sugar and fat absorption mechanisms and effects

Your body can use glucose as energy readily, but fructose can only be worked as fat or in the liver.

  • When the liver is flooded with too much fructose, too quickly, it is overwhelmed and converts most of the fructose into fat, triglycerides. Some of this fat fills up the liver and causes inflammation, which blocks the action of insulin in the liver.
  • Chain reaction occurs, in that the liver releases stores of glucose into the bloodstream, driving the pancreas to release more insulin, shuttling extra glucose and fat into cells. The rest of fat created by the liver is dumped into the bloodstream, where it ends up in fat cells, your arteries, and other potentially bad places.
    • Visceral fat is more hormone sensitive than other fat cells, resulting that its stores are dumped fat more readily than other fat cells, and since it sits close to the liver, it tends to clog up the liver and constrain its glucagon function when it dumps fat.
  • Sugar is 50 percent fructose; fruits also have it. An apple for example has 13g of sugar. 30 percent is glucose, the rest is fructose.

Comparing an apple to a highly processed fruit roll, apart from nutritional differences:

  • Fiber covering the apple (the skin) prevents the full absorption of sugar. Fiber makes food stay longer in the stomach, generating signals that release appetite suppressing hormones.
    • Speed of eating and amount of fiber is important. That’s why salads and low glycemic foods should come first in your meal
  • An apple gives your pancreas time to realize how much insulin to release. On the other hand, high glycemic, calorie dense foods are easily and quickly absorbed in the digestive process, prompting the pancreas to produce a lot of insulin, which it often overshoots, plunging blood sugar levels, which in turn makes us hungry.

Height as a measure of extra caloric availability and health

  • Maximum height is constrained by genes, but available energy will limit actual height. Because if you spend most of your energy fighting diseases and toiling in the fields, then you won’t have extra calories to gain height.
    • It was seen for populations that turned to subsistence farming, like in Asia and America, to have had their height shortened. For example, subsistence farming children spent about 4 or 6 hours working, as compared to children of hunter gatherers that would spend about 1 to 2 hours working in tasks like gathering firewood and helping in domestic tasks.
  • Tall people are naturally selected in hot climates, because of higher surface area to release heat. Colder climates select for shorter ones, because they can retain more heat.

Too much Stress

  • Stress is an ancient adaptation to save you from dangerous situations, and activates energy reserves when you need them.
    • If a Lion roars nearby, a car nearly runs you over, or if you go for a run, your brain triggers your adrenal glands (which sit on top of your kidneys), to secrete a small dosage of the hormone cortisol.
  • Cortisol does not cause stress. It is released when you are stressed.
  • One of cortisol’s functions is to cause liver and fat cells (specially visceral fat cells) to release glucose into the bloodstream. It increases heart rate and blood pressure, makes you more alert and prevents sleep, and makes you crave for calorie rich foods.
    • Chain reaction for long bouts of stress: as you crave more energy rich foods and consume them, this increases insulin levels, which inhibits the brain’s response to leptin (fat cells release it to increase satiety), as such, the stressed brain thinks you are starving, so activates reflexes to make you hungry and other reflexes ot make you less active.

Too little Sleep

  • Insufficient sleep promotes stress and stress promotes insufficient sleep. Higher income populations get on average more sleep, likely because of having of less challenges of making ends meet and less stress
    • Sleep deprivation is sometimes caused by elevated levels of stress, thus more cortisol.
    • Sleep deprivation elevates the ghrelin hormone, a hunger hormone that stimulates appetite.
  • Only until recently did we do start sleeping alone or with a single partner the child detached from parents during sleep, and in stimulus deprived environments
    • On the other hand, hunter gatherers used to sleep in relatively noisy environments, woke up at about 7h, took a one hour nap in the middle of the day, and then went to sleep at 9pm, where they would get two sleeps. Likely an adaptation, that prevented predator under vigilance
    • But now we have entertainment and lights that are able to keep us awake far beyond 9pm, in sensory deprived environments, with less exercise, excess calories, and stress

Too Crowded

Villages, cities, are our human fortresses from the wild. Because we have that fortress, we don’t need to carry arms and be attentive to outer animal threats, like lions. Mostly. There were costs to these though:

  • High concentration of humans in low4 sanitation5 environments facilitated the spread of these diseases, and close contact with animals gave rise to diseases such as influenza, coming from pigs.
  • Before the 1900s, the death rates in cities such as London were higher than in rural areas, so they needed a large influx of rural immigrants to keep them going. Which they did, because of a larger scope of opportunities and wealth.
  • Cows’ digestive systems were adapted to eat grass, but in the crowded industrial complexes they are fed grains, and as a consequence they have to be constantly medicated with antibiotics, to counter their chronic diarrhea and diseases coming from these crowded, filthy environments.
    • Antibiotics given to industrially raised animals because microbes also consume energy. So this makes them fatter

Cancer

There is a positive correlation between cancer development in reproductive organs and high energy balance.

  • Women’s bodies adapted towards delivering as many children as possible, and are maladapted towards having a high energy balance:
    • High exposure to estrogen incentives cell division in breast, ovaries, and uterus, in preparation of a fertilized embryo. Chances to develop cancer in these organs increases substantially from the number of menstrual cycles, and decreases as the number of children the woman bears.
      • During the menstrual cycle the levels of estrogen and progesterone rise significantly
      • While nursing, chances of having cancer go down, as there is less exposure to reproductive hormones, and likely breast feeding aids flushing memory ducts.
    • For women, the incidence of breast cancer was observed to be higher in nuns (hence, the nuns disease)
    • Women in contemporary USA have about 350-400 menstrual cycles, start menstruating at 12, 13, and then stop at early fifties
      • Hunter gatherers have about 150 cycles, and start menstruating at 16. They spent most of their life being pregnant or nursing, while barely having energy to do so.
    • Obese women can have 40 per cent more estrogen, because of the role of fat in the endocrine system. So after menopause being obese is correlated with cancer in women
  • For men, higher exposure to testosterone throughout their life is correlated with prostate cancer. But correlation is not as strong as in women.

More exercise results in lower chances of cancer. Likely because the energy spent there is not otherwise spent on reproductive hormones that have these side effects.

  • There is compelling evidence that routine physical activity is associated with reductions in the incidence of specific cancers, in particular breast and colon cancer. Physically active men and women exhibited a 30%–40% reduction in the relative risk of colon cancer, and physically active women a 20%–30% reduction in the relative risk of breast cancer compared with their inactive counterparts.

Use it, or lose it; No strain, no gain

Imagine you are tasked with building a robot that is aimed to accomplish a task in the future that is unknown. You can either create a series of specialized robots (e.g. only able to swim, to dig, jump, etc), or a single one that adapts to multiple situations. When you don’t know what the robot will do, the latter works best. That is how animals and plants work. But if you don’t use it, you lose it.

Bone loss

  • Bones are a reserve of calcium, an essential component to the body. If there is not enough calcium, osteoclasts start dissolving them at a higher rate.
  • Most people will reach their peak bone mass between the ages of 25 and 30, then it keeps going away. Specially struts in spongy bones like vertebrae and joints. They don’t come back.
  • Just having a supply of calcium and vitamin D is not enough. You also need to load your bones, otherwise the osteoblasts will not kick in to start building bone.
    • High intensity weight bearing can stop or even moderately reverse some loss.

Teeth problems

If you don’t chew enough when you are young, teeth will become misaligned, and overcrowding happens, leading to issues on wisdom tooth growth for example. The body needs that stress.

Allergies and hygiene hypothesis

Hygiene hypothesis states that early childhood exposure to particular microorganisms (such as the gut flora and helminth parasites) protects against allergies by properly tuning the immune system. In particular, a lack of such exposure is thought to lead to poor immune tolerance. There are two main hypothesis for it:

  • First hypothesis is that T helper 1 cells are not busy enough6, which increases the number of T helper 2 cells. With more sterilized environments, children’s immune systems are less busy, having more of these T helper 2 cells swimming unemployed, leading to a higher probability that they react to a non-threatening component. First reaction is mild, but the immune system keeps a memory, so the following time that component is detected, it’s devastating.
  • Second hypothesis is microbiome destabilization, the “old friends” hypothesis. Microbes have evolved to cohabit with others and us in a cold war kind of environment. Once you start killing several of them with antibiotics and cleaning products, you destabilize their equilibrium
    • There is a case to be made that possible treatments would be to have fezes or filth
    • And it also follows that after antibiotic treatments, that probiotics are taken

We frequently mistake comfort for wellbeing

Hannah Arendt introduced the expression and concept of the banality of evil, where a common person does nefarious actions, since they became accepted and normalized in their society.

Actions that we perceive as normal, such as usage of cancer inducing sodium nitrate in foods (which stems for a trade-off between structural economic gain and health), wearing comfortable shoes, reading and sitting, are in fact not normal when seen through the lenses of long term evolution.

Comfortable shoes

Our feet have adapted for us to efficiently run and walk, but highly cushioned, constraining shoes deform and inhibit our feet from fulfilling their purpose

  • Habitual barefoot people have much lower incidence of flat foot, which very likely is due to the high usage of the foot arch, which is restrained on comfortable shoes with arch support, which instead overload the plantar fascia, leading to plantar fasciitis
  • A constrained toe box restrains how much stability the foot can provide, and an unbalanced foot leads to an unbalanced knee, hip, body.
  • I’ve personally transitioned to using barefoot shoes daily and when hiking, and can only recommend them after using them for more than one year.
    • One common question is if these are appropriate on hard terrain. Turns out that your foot already has a built in mechanism, which is to forefoot / midfoot strike, instead of heel strike. Try to jump to see how your foot automatically lands using the ball of your foot, and how softly you land, when compared to a (dangerous) similar jump landing on your heel.
    • Going barefoot or using minimal shoes, provides you full perception of the terrain and impact, so you’ll adapt towards hitting the ground more softly, and not using heel strikes as much.

Reading

Myopia is a formerly rare evolutionary mismatch that is exacerbated by modern environments. In the USA and Europe, about one third of all children (aged 7 to 17) become nearsighted.

Evidence suggests that being nearsighted used to be very rare: that less than three per cent prevalence amongst hunter gatherers and populations that practices subsistence agriculture. In 1813, it was noted that amongst the Queen’s guards, many were myopic. On the hand, from the 10k foot guard, less than half dozen were myopic.

The mechanisms of how myopia are still not fully understood, but two main causes are suggested:

  • Close work, forcing long bouts of close focus. Singapore study found that students who read more than 2 books per week had strong myopia.
  • Children spending more time outdoors have lower incidence of myopia. Brightness of light and visual stimuli appear to have a beneficial effect.

Sitting

While sitting in different positions to rest after strenuous activity has been around for much of our species, only until recently have we been multiple hours sitting in a chair, day after day, which is a risk factor towards:

  • Muscle atrophy and shortening: calf muscle shortening also happens for high heels usage
  • Lower back pain. To prevent this issue, make sure to maintain a strong back via moderate intensity load, but not too much load, like furniture movers.

How to solve these issues

One solution would be to wait for natural selection to sort out these problems, which is very unlikely due to the high rate of change in our societies and environments, in short time windows.

As the commander of the Albigensian Crusade said, on a mission to eliminate heretics: “Kill them all; let God sort them out.”

We are all subject to influence by advertising, availability and peer pressure, but we can be nudged to acquire behaviours that benefit us. Daniel Lieberman suggests:

  • We should invest in prevention over treatment
  • Although adults have the right to get sick, children need guidance. How different is it to restrict fast foods to them, to restricting which movies they are able to attend? Or availability of alcohol to them?
  • Just like nature obliged us to do the right thing and evolved our behaviours, government has the right and duty to have information to do rational decisions and nudge or even push us to do so:
    • Question is how much and where and when
    • Government shouldn’t prevent you from smoking, but “you are free to do as wish, as long as I don’t have to pay for it”

Culture does not allow us to transcend biology

Culture is roughly everything we do and monkeys don’t. ― FitzRoy Richard Somerset

Clever as we are, we cannot modify our human bodies any more than superficial ways. And it’s arrogant to think we can engineer body parts any better than nature did. There will be no Pasteur for mismatched diseases.

We should come to terms that:

  • We are fat, furless, bipedal primates that crave sugar, salt, fat and starch.
  • We crave comfort, but our bodies are endurance athlete machines.
  • As Voltaire wrote: “Let us cultivate our garden”. We must cultivate our body.

  1. Compared to other animals with the same body weight, they have a fifth of the brain size, and double the intestine system. Size of the brain correlates with the size of the group. Humans are able to interact with 100 to 200 individuals.↩

  2. Choking on food is the fourth leading cause of accidental death in the USA.↩

  3. Between 1985 and 2000, the purchasing power of the US dollar decreased by 59 per cent, the price of fruit and vegetables doubled, fish increased by 30 per cent, diary about the same. In contrast, sugars and sweets became 25 per cent less expensive, fats and oils 40 per cent, soda 66 percent less expensive. Portion sizes ballooned: in a fast food restaurant in 1955 order of hamburger and fries would yield 412 cals. Today for the same inflation adjusted size, the same order would have double the calories, 920 cals.↩

  4. If you are curious of how the sewers of earlier cities smelled, the Paris Musée des Égouts provides an illustrative tour through these sights and smells ↩

  5. Thomas Crapper did not invent the toilet. But was a pioneer in its mass manufacturing. Origin of the word “crapper”↩

  6. For example, Hepatitis A virus stimulates T helper 1 cells, which suppress the number of T helper 2 cells↩

]]> 2025-02-10T00:00:00+00:00 2024 Lookback https://lopespm.com/notes/2024/12/31/2024-lookback.html https://lopespm.com/notes/2024/12/31/2024-lookback.html It’s time to wrap up and celebrate 2024, an eventful year that fueled my yearning for internal growth and curiosity, with challenges and lessons I am grateful for.

I’ve recently built a set of Grafana dashboards to track my personal goals, which I’ve been finding incredibly interesting to explore and visualise (something that might be interesting to share in a later post).

Final Tally

There are other metrics I track, but specifically to 2024 content creation, these are the final statistics1:


Notice that the tracked metrics are on the volume of content produced, not their number of views, subscribers, likes. Although keeping track of topline metrics makes sense for a full fledged business, I chose instead to track what I can control, which is my output. At the end of the day, this is a hobby, a labor of love.

If I can influence someone’s life for the better, that will already fill my heart to the brim. This, added to how important posting these contents has become to consolidate and structure my thoughts, and serendipitous opportunities they randomly opened, make this an incredibly satisfying endeavor.

I’ve experienced time and time again that consistency eventually bears wonderful fruits. Who knows which doors might open one day later? I don’t, but I’m curious to find out.

2025, one day away

With 2024 wrapped up, let’s welcome 2025. I’ve pinned down next year’s goals and will be looking forward to sharing what I learn along the way. Happy new year, and see you on the other side! 🫡

  1. This dashboard was produced using a custom Grafana dashboard, where the RSS feeds for this blog and YouTube channels were tracked, transformed and presented in simple gauge widgets↩

]]> 2024-12-31T00:00:00+00:00 Note Taking https://lopespm.com/notes/2024/12/30/note-taking.html https://lopespm.com/notes/2024/12/30/note-taking.html


Over the past 4 years, I’ve written about 1820 pages on the 13 A5 notebooks shown in the picture above. This tool has become essential for my job, and in this article I’ll describe the simple process behind it, which I’ve found to deliver consistent results, reliably.

Write, Consolidate, Repeat

The process I’ve been using for several years is based on Option 3. of Solving Task Switching Through Documentation, and is broken down in three steps:

1. Write

  • Material: A5 spiral notebook (it makes it easier to be written anywhere, because the pages can always be flipped around in a way that only the writing page is facing up, and the rest of the pages serving as a solid writing surface; plus the notebook’s rings are great for carrying your pen or pencil reliably), with clear pages (this allows for diagrams, phrases, texts, tidbits, collaborations to happen without any formal restriction). Have another backup notebook within reach, to replace the main one when it runs out.
  • When: Make sure to always have your notebook and pen / pencil at your desk, meetings, conferences, 1:1s, and any environment where a collaboration is made, within reason.
  • What: Write down your thoughts, key points that coworkers communicate, ideas, anything else you find relevant. They are your notes, so allow yourself to expand and experiment.

2. Consolidate

With several notes written, their consolidation can happen in three different time spans:

  • Immediately:
    • The mere act of writing often helps the thought process, since it forces ethereal concepts to be materialized into paper. Expect that a good portion of the written notes to not be read ever again.
  • Shortly after they are written:
    • Keeping track of conversations: has it ever happened to you that suddenly you completely lost track of the discussion, especially a complex / ambiguous one? Refer back to the jotted down key points to quickly recall and connect points. Soon after you will be back on track, and maybe with new insights.
    • Summarization of conversation segments, usually at the end of a meeting: leverage the written notes to summarize key takeaways and action points. There is nothing worse than spending 30 minutes in a discussion without material end results or follow ups.
  • Some time after notes are written: I avoid this period to be longer than one day.
    • Transcribe key points into a structured medium, such as a document, article, or personal log (I use a simple text file where I log important events during the day; this text file has been consistently updated for 4 years, and is a personal historical treasure trove spanning several thousands of lines).

3. Repeat

Just like other processes, their value only becomes apparent after they have been used several times. Build a habit. It will then become immediately apparent whenever a notebook is not at hand, or when structured notes are not written down.

]]> 2024-12-30T00:00:00+00:00 Diary of a CEO Book: Lessons https://lopespm.com/notes/2024/12/28/diary-of-ceo-book-lessons.html https://lopespm.com/notes/2024/12/28/diary-of-ceo-book-lessons.html I’ve recently finished reading Diary of a CEO: 33 Laws of Business and Life, which draws from both Steven Bartlett’s entrepreneurial journey and interviews from the homonymous podcast. These are are my main takeaways:

Building high performance groups

Incentives and culture

  • If you want to predict what a group of people will do over the long term, you need to look at their incentives, not their instructions.
  • Companies don’t have one company culture: every manager in an organization creates a subculture underneath them.
  • In a small group / startup, when the culture is strong, new joiners become like the culture. When the culture is weak, the culture becomes like the new joiners.
  • In a small group / startup (and likely this can be applied to teams), create a cult-like dynamic upon its inception, but then transition it to a more sustainable structure. Key elements of a cult:
    • Provides meaning, purpose and belonging. Asserts superiority of the group.
    • Clear shared identity and commitment to ideology.
    • Leaders present themselves as infallible, confident and grandiose.
    • “Us vs Them” mentality: there is a clear adversary.

Traits of a good leader

  • The factor that kills meaning the fastest is a leadership team that dismisses employee’s work or ideas, removes their sense of ownership and autonomy, and asks them to spend time on work that is cancelled, changed or disregarded before it’s been completed.
  • Allow people the space to fail and succeed. The main job of a leader is to be a supportive enabler, not a critical micromanager.
  • Leaders should proactively remove any obstacles, bureaucracy and sign-off processes that prevent the team to achieve daily progress, this includes identifying and providing the required resources for them to do their job. To discover these gaps, ask the teams informally or formally about these problems (via retros for example).
  • Leaders need to point out, publicize and praise progress as loud, far and wide as they possibly can. Recognition reinforces behaviour, but it also acts as evidence to adjacent teams that progress is possible for them too.
  • Alex Ferguson, the widely successful manager from Manchester United, didn’t care about tactics, strategies and formations. He cared about getting the best out of each individual, the team’s culture and attitude. He didn’t want them to be become complacent:
    • He had different ways of dealing with different players. He knew how to get the best out of everyone. He knew his players well, and what made them tick, like Gary Neville’s important connection with his grandparents.
    • Some players strived under Ferguson’s “hairdryer” treatment, others under a more compassionate approach, and others by being more hands off. There was not a size fits all approach. The main objective was always to have the team and club at a higher ground.
    • It is impossible to seamlessly blend into a team as a jigsaw piece, unless you comprehend the unique shape of each of your team members.

Ask who, not how

  • Every CEO and founder will be judged on their ability to hire the best, and bind them into a culture that gets the best out of them. An environment where they become more than the sum of the parts.
  • Your ego will insist that you do. Your potential will insist that you delegate.

Excise bad apples, promote bar raisers

Dealing with discomfort

  • “Hard” is the price we pay today for an “easy” tomorrow.
  • When you refuse to accept an uncomfortable truth, you’re choosing to accept an uncomfortable future.
  • When something is unresolved because we’ve chosen to bury our heads in the sand, it doesn’t sit dormant, waiting to be addressed; it becomes toxic, contagious and poisonous to those around us, inflicting more collateral damage with every day that it remains unaddressed.
  • When seeking the truth that is causing the discomfort, listen to understand, not to look for a victory, but rather in the perspective of a partner that wants to overcome that difficulty.
  • In a relationship, if you’re having the same conversation over and over again, you are having the wrong conversations. You’re avoiding the uncomfortable conversation you should be having.
  • It is in talking about our disconnections that people create more connectedness with one another.
  • You can predict someone’s success in any area of their life by observing how willing and capable they are at dealing with uncomfortable conversations. Your personal progression is trapped behind an uncomfortable conversation.
  • As mentioned by George Vaillant, denial can be healthy, enabling individuals to cope with rather than become immobilized by anxiety, or it can be unhelpful, creating a self deception that alters reality in ways that can be dangerous.
  • “People think they’re motivated by seeking pleasure; they’re wrong, they’re motivated by avoiding discomfort” - Nir Eyal

Pressure only comes to those who earn it

Building good habits

  • Instead of fighting against a bad habit, acknowledge the cue, routine, reward habit loop. When removing any of these elements, its void will need to be replaced. Replace rather than delete.
  • Cues are powerful, since they are the starting points to routines. As such, be mindful of how you set up your environment. For example, Steven Bartlett placed his DJ set in the kitchen table, in plain sight, and made sure to take only one click to start practicing, in order to reduce perceived cost of practicing it. I’ve personally done the same thing with my camera setup to shoot YouTube videos, by using a simple and predictable set up for video and microphone, which I have with highly visible in my living room, incentivizing me to do more videos.

Persistence

Incremental wins, keeping momentum

  • How much you are achieving is pretty much irrelevant to your motivation: but if you feel like you are getting somewhere, you’ll be driven to keep going.
    • When problems seem insurmountable or too hard or unknown, that’s when procrastination creeps in.
    • The key here is to break down the problem into smaller, more manageable ones. The small resulting improvements and the sense of momentum will keep you going, and these will eventually compound.

Applying time and energy effectively

  • The exact same skill applied in a different industry or context can have a widely different value, as seen by a Washington Post experiment where one of the world’s greatest violinists Joshua Bell, who was able to professionaly command $1,000 a minute, barely got any attention or spare change from playing in the subway.
  • Position yourself correctly, and get the skills that your industry finds to be rare and valuable, and that your competitors don’t have.

Time

  • Time betting exercise
    • Imagine you, and everyone is sitting around the game of life. Each player is assigned a number of chips, each representing an hour. They don’t know how many they have. Maybe it’s one, maybe it’s 500k. You can try to predict, but you don’t really know.
    • The rule is that every hour you need to put one chip on the table, or it will just be placed automatically. You can place it on Netflix, learning a new skill, sleeping, etc. Certain actions that increase your health will increase your stock of chips.
    • Your fitness will block you from betting on certain positions. For example, if you have a serious locomotion injury, you cannot choose to run.
    • The game ends when all chips are gone. You don’t get to keep anything you earned during the game.
  • Being selective about how you spend your time, and who you spend your time with, is the greatest sign of self respect.

Failure as an advantage

  • Ask the questions: Why will this fail? Why is this a bad idea?
  • Apply the pre-mortem process to visualize and attempt to preempt failures in your plan. This will mitigate risk and likely increase your prediction accuracy.
    • A 1989 study found that prospective hindsight—imagining that an event has already occurred—increases the ability to correctly identify reasons for future outcomes by 30%.
    • Steven Bartlett’s pre-mortem method:
      • Set the stage: the objective of the exercises. It is to uncover risks, not to look for escape goats.
      • Fast forward to failure. Ask team to visualize failure as clearly as possible.
      • Brainstorm reasons for failure.
      • Ideate independently, and then share and discuss.
      • Develop contingency plans.
  • Share experiment failures, so that their knowledge can be leveraged when building new experiments.
  • “I have not failed, I have just found 10k ways that won’t work” - Thomas Edison
]]> 2024-12-28T00:00:00+00:00 Feeling seen https://lopespm.com/notes/2024/12/21/feeling-seen.html https://lopespm.com/notes/2024/12/21/feeling-seen.html


This ad struck me as the culmination of a long-standing trend. I struggled to articulate it at the time, but I’ll attempt to do just that in this article.

Let’s start by the ad itself. It carries implicit symbols and memes, whose interpretations will vary wildly from person to person, but in general I believe it attempts to convey these subtle messages:

  1. In order to be seen and acknowledged as an individual, one must spark admiration and rise above the crowd with a stylish look, makeup and fashionable garments. Appearance is the solution.
  2. Feeling seen is about exterior validation (which requires the advertised beauty products), rather than real understanding and connection with another being.
  3. You need to command the attention of acquaintances and strangers, their judgement is important for your self worth.1

Relying only on external validation, specially from foreign entities, as a means to sustain one’s self-esteem is a time honored strategy to produce nefarious results. Yet, it’s celebrated by certain influencers, mainstream media, TV shows, ads and movies that attempt to distort a small sample into a normality.


Demanding Standards

When external validation is postulated as a basic human need, and one’s default natural, vanilla body is not sufficient to garner the validation one needs (as the above ad suggests), it soon follows that external interventions are required to achieve it.

I was fortunate enough to grow in an environment where artificial external displays of vanity were seen as a quirk, and sometimes even frowned upon (depending on the situation), thus not a requirement for a happy and fulfilled life. But once one is born or dislodged into the eye of a hurricane, it is hard to see the surrounding damage, especially when peers abide by those standards, generating an underlying group integration peer pressure.

Several off-the-shelf beauty products are known for their toxicity, contributing to increased prevalence of cancer and infertility, yet they stand as one of the cornerstones of modern standards of beauty.

How we show up is important, and is a meaningful way of self expression, but when the baseline for allowing one’s self expression requires one to consume / apply several products (sometimes on the go), or even subject to invasive body modifications, then I would argue that something is wrong in the underlying societal system.


Mind the ouch

One such body modification, turned mainstream, is the injection of botox.

Botulism is an often-fatal nervous system disease, first recorded in Europe in 1793, claiming the death of over half of those patients who had become ill through eating uncooked blood sausages. botulus being the Latin word for sausage. One one-millionth of a gram this neurotoxic protein can kill a man, and one pint would be enough to kill everyone on earth.

In 1989, purified botulinum toxin was approved as a drug under the brand name Oculinum, and later renamed to Botox. In small doses, the same nerve damage that causes fatal paralysis in poisoning cases, helps to remove forehead creases and crow’s feet, with the only side effects being an inability to express emotion using your face, and an occasional case of drooping eyelids.

Today, injecting the deadliest substance on earth into one’s face is normalized, as seen in this TV commercial (where its mid-commercial disclaimers makes one question if it’s a real ad, or a parody2), or as seen in this suggestive London tube ad:


Look Within

I hold nothing against those who choose to subscribe to a certain mode of expression or style. I too, have my own. I do welcome you to introspect why you chose it. If your sincere why comes from a deep rooted desire to feel accepted and valued by others (good to receive these in healthy dosages, but dangerous when overbearing), then it might be something to look out for.

Relying solely on external validation likely results in feelings of emptiness and insatisfaction. I’ve been there several times, and its not pleasant. Instead, attempt to obtain self validation by looking in, rather than out. Some simple, yet powerful strategies:

  1. Consider your ancestors, a stable past. For example, what would your ancestors eat, do, that worked out for their health? Our bodies have adapted throughout milenia to certain patterns, and if we are on their descendancy line, surely they did something right. Time is the great filter of nonsense and fallacies.
  2. Recognize the cycle: ask yourself why you are seeking external validation, and understand the patterns and needs that lead to it.
  3. Practice self-appreciation: build your self-acceptance structure, practice gratefulness and appreciation for what you are and did (a journal might help).
  4. Rejection therapy: put yourself in situations you seek rejection. Start small, such as asking for a discount or applying for a job you think you are not qualified for. It helps you realise that rejection is not as bad as it appears, which decreases the need for other’s validations.
  5. Set personal goals: instead of relying on others to measure against, take control, and set goals that serve your needs and desires. Keep yourself accountable to hit them.
  6. Listen to your intuition: you know yourself best, so why rely on others for answers? Reach within and take distraction free time to listen to yourself.
  7. Self care to self validate: do activities that make you feel good, use self affirming thoughts, take care of yourself.

  1. Why: friends, family and close connections likely already acknowledge you by who you really are, or at least should. Why would they need you to wear a beauty product in order for you to feel seen? Therefore the remaining target audience are acquaintances and strangers↩

  2. Transcript from the TV commercial: (…) FDA approved to temporarily make frown lines crow’s feet and forehead lines look better. The effects of Botox Cosmetic may spread hours to weeks after injection causing serious symptoms alert your doctor right away as difficulty swallowing, speaking, breathing, eye problems, or muscle weakness may be a sign of a life-threatening condition. Do not receive Botox Cosmetic if you have a skin infection. Side effects may include allergic reactions, injection site pain, headache, eyebrow, eyelid drooping and eyelid swelling. Tell your doctor about your medical history muscle or nerve conditions and medications including botulinum toxins, as these may increase the risk of serious side effects. So get that “just saw a puppy look”! (…)↩

]]> 2024-12-21T00:00:00+00:00 48 Laws of Power: Lessons https://lopespm.com/notes/2024/12/08/48-laws-of-power-lessons.html https://lopespm.com/notes/2024/12/08/48-laws-of-power-lessons.html I’ve recently finished Robert Greene’s 48 Laws of Power, a controversial, yet useful book. The information it shares makes one better prepared to navigate our complex and perilous world of influences, power and subtle manipulations.

I found it to be an eye-opener on some fronts, but mostly as providing different perspectives on things that on the surface, appear obvious.

I can recommend this book to just about anyone. Its information, just like a knife, can be used for useful, enriching and productive endeavours (like cutting vegetables for a family dinner), or for nefarious activities (such as inflicting harm upon others). It’s ones responsibility to use it correctly and morally.

Below are my key takeways, segmented by Behaviour, Resources, Relationships, Strategies and How you Show Up.

Behaviour

Attitude and Behaviour

  • Be royal in your demeanor. How you present and carry yourself tells the people how they should treat you. As seen in Columbus, son of a cheese maker, who behaved like royalty, as someone who deserved to be treated well. He amazed the Portuguese king, and eventually got the Spanish royalty to get him what he wanted: a fully paid expedition, except for a lifetime 10% profit shares for him and his descendants. They felt like he was one of them. Even though he knew less about navigation than any of his sailors, was a bad leader, etc, he still got what he wanted, because of how he carried himself.
  • Beware of venerating a new seductive culture that is not your original one. Don’t despise and look down upon the culture that raised you and provided you gifts, for it will grow resentment of the ones that follow it.
  • Boldness over timidity. Both are acquired behaviours. Timidity gives others time to think and plan. Makes even the tamest prone to attack when they sense blood in the water.
  • Only the weak rest on their laurels and dote on past triumphs.

In victory, don’t go past your mark

  • Success is intoxicating and dangerous. The moment of victory is the moment of greatest peril. Do not allow success to come to your head. Set a goal, and when you reach it, stop.
    • Know when to stop and consolidate wins, instead of persisting in pushing forward.
    • When success is attained, step back, and reflect upon the conditions that led to it. Don’t simply repeat the same actions again and again.
    • History is littered with corpses and fallen empires who did not know when to stop.
    • There is a Japanese saying that Furuya Sensei was fond of, “Katte kara kabuto no o wo shime yo” which means After victory, tighten your helmet. Never let your guard down even if you think you have won. That just might be what your opponent wants you to think.
    • The moment when you stop has great dramatic importance. What comes last sticks in the mind as an exclamation point. Best time to stop is after a victory. Keep going, and it will lessen it’s effect, or even reverse it.
    • “Nothing can be more important than to close your examination with a triumph. So many lawyers succeed in catching a witness in a serious contradiction; but, not satisfied with this, go on asking questions, and taper off their examination until the effect upon the jury of their former advantage is lost altogether. “Stop with a victory” is one of the maxims of cross-examination” - Francis L. Wellman, The Art of Cross Examination
  • Bad luck teaches you patience, timing and the need to be prepared for the worst. Good luck deludes you into making you think your brilliance will carry you forward. And when inevitably misfortune comes, you will lack adequate preparation.
  • As it’s taught in riding school: you have to be able to control yourself, before your can control the horse.
    • And even when you can control yourself, there will be people forcing you to push forward. Be careful, and manage them. Feed them with small victories, but don’t allow yourself to engulfed in the momentum and go all in.
  • “A man famed for his tree-climbing skills once directed another to climb a tall tree and cut branches. While the fellow was precariously balanced aloft, the tree-climber watched without a word, but when he was descending and had reached the height of the eaves the expert called to him, ‘Careful how you go ! Take care coming down !’. ‘Why do you say that ? He’s so far down now that he could leap to the ground from there,’ I said. ‘Just so,’ replied the tree-climber. ‘While he’s up there among the treacherous branches I need not say a word – his fear is enough to guide him. It’s in the easy places that mistakes will always occur.’ Lowly commoner though he was, his words echoed the warnings of the sages.” - Yoshida Kenkō, Essays in Idleness

Don’t appear too perfect. Don’t boast.

  • By making others aware of their inferior position, you are only stirring an unhappy admiration, this is, envy.
    • To show envy, is to admit your inferiority, and that is why people don’t normally admit it.
    • “Of all the disorders in the soul, envy is the only one no one confesses to.” - Plutarch
    • Cosimo de’ Medici avoided such issues by being simple on the outside, and only showing his wealth, elegance and opulence on the inside (of his house, for example).
      • “There is in the garden a plant which one ought to leave dry, although most people water it. It is the weed called Envy.” - Cosimo de’ Medici
  • Kierkegaard believed that the creator of envy is as much to blame as the person who feels envy. Don’t crow about your victories and superiority.
    • The naturally perfect are the ones that need to work the most to hide their features. Money and power are available to be gained, but not natural beautiful features, good looks, sharp natural intellect, etc.
  • A sudden increase in power, promotion are dangerous. Downplay them, don’t flaunt them, and attribute them to luck, since luck is possible to all.
  • Be careful of false modesties that can be seen through. Put on a good act. Otherwise it will only create more envy.
  • People cannot envy the power that they themselves have given a person.
    • When Ivan the Terrible died, Boris Godunov knew he was the only one on the scene who could lead Russia. But if he sought the position eagerly, he would stir up envy and suspicion among the boyars, so he refused the crown, not once but several times. He made people insist that he take the throne. George Washington used the same strategy to great effect, first in refusing to keep the position of Commander in Chief of the American army, second in resisting the presidency. In both cases he made himself more popular than ever.
  • “Envy is the tax which all distinction must pay” - Thoreau
  • Pay a high price if needed, to keep envy in check.
  • “It takes a great talent and skill to conceal one’s talent and skill” - La Rochefoucauld

Rationality and Control

  • “Wise men [should be] like coffers with double bottoms: which when others look into, when opened, they see not all that they hold.” - Sir Walter Raleigh
  • Petulance is not power, it is a sign of helplessness.
  • In the face of a hot headed enemy, an excellent response is no response.
  • Nothing is as infuriating as a man who keeps his cool, when others are losing theirs
  • Only use raging tirades when you are in control of them, and in a precise manner, and very rarely. They are a (dangerous) tool.

Use your words and actions wisely

  • Once your words are out, you cannot take them back. Keep them under control and be particularly careful about sarcasm. The momentary satisfaction you get from your biting words will be outweigh by the price you pay
  • “Oysters open completely when the moon is full; and when the crab sees one it throws a piece of stone or seaweed into it and the oyster cannot close again so that it serves the crab for meat. Such is the fate of him who opens his mouth too much and thereby puts himself at the mercy of the listener.” - Leonardo Da Vinci
  • “If you are not in danger, do not fight” - Sun Tzu

Be strategic in your approach

  • When in a position of power, don’t be completely predictable, as that could be exploited. when is a position of serving others, be careful when doing that, because it might make you be seen as unreliable, or even having a psychological pathology
  • Attach yourself to the real decision maker, not the people with titles. As seen for Richelieu, who attached himself to King Louis XIII’s mother, instead of the king himself, since she was calling the shots. As a result, Richelieu quickly rose through the ranks.
  • Winston Lord had worked on a report for days, which Kissinger handed back with the notation “Is this the best you can do?”. Lord rewrote and polished and finally resubmitted it; back it came with the same curt question. After redrafting it one more time–and once again getting the same question from Kissinger–Lord snapped, “Damn it, yes, it’s the best I can do.”. To which Kissinger replied: “Fine, then I guess I’ll read it this time.”
  • North wind and the Sun made a bet on who could make a man take off his coat. First, the North wind gushed one him, but that only made the man hold on to it more tightly. Then, the Sun provided a warm heat that made the man take off top coat, and then blazing heat which made the man strip and bath in the river. Persuasion is more effective than force.
  • “MAN: Kick him-he’ll forgive you. Flatter him-he may or may not see through you. But ignore him and he’ll hate you” - Idries Shah

Resources

Focus your energies

  • What’s concentrated, coherent, and connected to it’s past, has power.
  • Concentrate your forces: Intensity beats extensity every time. You gain more by finding a rich mine and digging it deeper, than by fleeting from one shallow mine to another
  • Dispersion can be a good strategy when your opponents are stronger.
  • An example caveat of concentration, is if the sole person providing you power dies or leaves the scene, you are in trouble, so to protect against that risk, you need different concentrated sources of power. As seen with Cesare Borgia’s transgression of this, who derived power mainly from his father, the Pope, who gave him armies to fight with. When his father died, he was as good as dead.
  • Do not spend time, attention or energy on things you cannot have. Your attention, or lack of, is a form of power. When King Louis wanted to punish someone on their courthe would just ignore them and cut them off
  • “I had far rather be confined to one element, and be admired in that, than be a Goose in all.”

Focus on the important things

  • Paranoia over small ticket issues often are destructive in the end. As seen when Kissinger made a recommendation to create a group to “plug the leaks” following the publication of the Pentagon Papers. The Plumbers, as they became known, eventually broke into the Democratic Party offices in the Watergate Hotel, setting off the chain of events that led to Nixon’s downfall.
  • When you make a mistake, treat it lightly. When a petty enemy attacks you, make none of it, show contempt. Show contempt publicly, but keep track of it privately. Don’t let a small problem or foe become a cancerous cell that gains power and overtakes you. Deal with them as appropriate.
  • “To disregard, is to win regard” - Italian proverb

The value of Time

  • “Space we can recover. Time, never” - Napoleon Bonaparte
  • Time is a human construct. Perception of time is subjective. Younger people feel like time is going slowly. Older people feel like time is whizzing by.
  • Long time where we wait patiently for the best opportunity (normally years), short time and end time (where we act decisively and with force
  • Three types of time: long, forcing and end time:
    • Long time: happening on a years time scale, and must be managed with patience and gentle guidance. Key is to not react impulsively, and wait for the right opportunity.
    • Forcing time: when people are indecisive, give them a deadline. Never give them time. As seen with Joseph Duveen, a successful art dealer who when dealing with indecisive clients, would mention that another tycoon would be interested, something happened and he had to leave the country, etc. This works as long as the party does not know what you are up to.
    • End time: when a plan must be executed with speed and force.
  • Patience is worthless if it doesn’t conclude with action. Waiting without doing action is a sign of indecision and procrastination. Timing is important.

Despise the free lunch

  • Nothing is more costly than something given free of charge.
  • What is worth, is worth paying for. Always pay the full price, there is no cutting corners to excellency, for bargains are often accompanied by a complicated psychological price tag.
  • Generosity is a sign, and a magnet for power. Keep money in constant circulation.
  • The powerful understand they need to protect their most precious resources. Indepence, time, energy and room to maneuver. By paying the full price they keep themselves free of dangerous engagements and worries.
  • Strategic generosity: give when you are about to take.
  • Avoid bargain demons (who only consider the explicit price tag, and not the cost in time, energy, dignity and peace of mind), the money sadists (who abuse their payer position power, by for example making you wait for a payment), the indiscriminate givers (who likely are looking qualm a emotional need, are emotional drains, and want to be loved; if they give to everyone, why should you feel special?)
  • The powerful must have have grandiosity if spirit, and should never reveal any pettiness. Money is the most visible arena in which to display either grandior or pettiness.
  • And if you want to meddle with the work of the creative people you hire, at least pay them well. Your money will more easily buy their decision, rather than your shows of power.
  • Powerful people give freely, buying influence, rather than things
  • A one time gift given at a sudden unexpected time, bears more force and impact. It will not spoil your children, but rather keep them under your thumb.
  • The more the money you spend on gifts and acts of generosity play with sentiment, the more powerful they are.
  • Don’t give in to greedy “El Dourado”s that will only exhaust your time and energy, with little to no return.

Relationships

Perils of Isolation

  • Do not create a fortress around you, it will make you detached from the world and make you an easy target, since it will be easy to know where you are, and be attacked.
  • Pontormo, a painter who spent 11 years secluded inside a chapel painting for his patron. Terrified of being copied by others and growing more and more afraid of fellow humans, he died, and his paintings / frescos did not survive him. Vesari, who saw his paintings, described them as being embodied by madness. Perspective flaws, too many characters, several compositional flaws, figures that overlap one another. Pontormo obsessed over details, and lost track of the composition.
  • Shakespeare was well known because many people knew about his works, he didnt work in isolation.
  • Isolation is good to gather one’s thoughts and escape the claws of conformity by society, but this should be done judiciously.
  • The more time you spend isolated, the harder it is to come back to society. Make sure you always have a way to come back to society.

Enemies, and Friends

  • Without enemies we grow lazy. Having enemies sharpens our wits. If you don’t have enemies, find them
  • Without a worthy opponent, a man, woman or group cannot grow stronger.
  • You are better having declared opponents, instead of not knowing where they lie.
  • You need people who deliver results, not that appease you.
  • Men are more ready to repay an injury than a benefit. Gratitude is felt as a burden, revenge as a pleasure.
  • “To have a good enemy, choose a friend: They know where to strike” - Diane De Poitiers, 1499-1566, Mistress of Henry II of France
  • A brahman, a great expert in Veda who has become a great archer as well, offers his services to his good friend, who is now the king. The brahman cries out when he sees the king, “Recognize me, your friend!”. The king answers him with contempt and then explains: “Yes, we were friends before, but our friendship was based on what power we had. I was friends with you, good brahman, because it served my purpose. No pauper is a friend to the rich, no fool to the wise, no coward to the brave. It is two men of equal wealth and equal birth who contract friendship and marriage, not a rich man and a pauper. An old friend—who needs him?” - The Mahabharata

Succession

  • The formulas used by the successful father worked well for him, because he had nothing to lose, and gave him his kingdom and fortunes. The son, who lives in a different set of circumstances, could have his father’s ways imposed onto him, in a domineering and oppressive way, imposing his lessons on the son. Instead of trying to put the son in a new direction, the father tries to put him on his own shoes, perhaps secretly wanting the son to fail, as he often resents his youth and opportunities. Resulting in the son to be cowed and cautions and fearful of losing what the father had gained.
    • Alexander the Great only wanted to outdo his father. To obliterate his name from history. He resented that his father conquered most of Greece, leaving not much for him to conquer. He wanted to out his successful father in the shadows, instead of the opposite. Which he did.
    • As a young rebel grows older, his struggle with his father often wanes, and he gradually comes to resemble the very man he used to defy.
    • The presence of the “father” must be constantly slain and kept in check. The old ways, the father, the comfort, are enemies of progression and power.
    • Keep an eye on the young, since they will do the same to you. Identify them and keep them under control.
  • It would seem easy for the son or successor to build on a grand foundation left for him. But in the realm of power, the opposite is true. The pampered indulged child, almost always squanders their inheritance, for it doesnt start with a need to fill a void. As seen with Louis XIV, XV, XVI. Louis XIV, while navigating through troubled times, built Versailles, instead of taking over his father’s palace, the Louvre, and reigned for (mostly glorified) 55 years. The last years of his reign had been difficult, but it was hoped that his child would develop into the kind of strong ruler who would reinvigorate the land and add to the firm foundation that Louis XIV had laid. Instead, his son, Louis XV gave himself over to pleasure. Worn out by debauchery, his country and his own finances were in horrible disarray. His grandson Louis XVI inherited a realm in desperate need of reform and a strong leader. But Louis XVI was even weaker than his grandfather, and could only watch as the country descended into revolution.
    • “Necessity is what impels men to take action, and once the necessity is gone, only rot and decay are left” - Niccolò Machiavelli
  • In Sumatra, the king would be killed ritually in a brutal fashion, in order to limit his boundless power and give space for the new generations.

Strategies

Strike the Shepherd, and the sheep will disband

  • Athenians found an effective way to deal with socially undesirable individuals. They didn’t fight, punish, or re-educate them. They would ostracism them.
  • Cancer begins with a single cell. Be quick and swift to isolate bad seeds from the group, before they infect a critical mass.
    • It’s better to isolate them, rather than destroy them. It’s less brutal. For in a game power, isolation spells defeat. Ostracize, limit their power. Spare no time arguing with them.
    • “If an injury has to be done to a man it should be so severe that his vengeance need not be feared” - Niccolo Machiavelli, The Prince
    • Swindlers isolate their targets from their normal social circles, making them more vulnerable to influence and manipulation. Apart from the void left I’m the sheep
    • Lure the poisonous person away, while staying away from his attacks, as seen with Pope Boniface’s isolation of Dante away from the “Whites” and leaving a power void on their faction. As a result Florence was taken over by the “Blacks”.
  • “When the tree falls, the monkeys scatter” - chinese proverb
  • Alternate harshness with mercy to soften them up. Play on their fears, but also their loves. Target primary emotions: Love, hate, jealousy.

Mirror effect

  • Dr. Milton H. Erickson, when faced with a couple with problems in bed, observed the same dynamics when they mentioned how they liked to dine. He didn’t address it directly, but instead told them to schedule a dinner where both of their desires would be met (for one desired a slow pace, and the other a big meal). It worked as expected, and they transferred the lessons acquired from the dining table later on into their bed.
  • Addressing head-on people will often lead to conflict. Instead, mirror their actions so they can reflect on their own, and they are to fix themselves without admitting fault. They will think they did it on their own, without anyone realising it. But be careful when being too obvious, as it will make them think you are manipulating them, thus undermining your efforts.

Reformation Perils

  • People know that change is needed, and are ok with superficial changes, but are deeply troubled when the changes are deep and affect their core beliefs, structures and habits.
    • Change is upsetting to the human animal, even when it’s for the good. If change is needed, find ways to disguise it, and sweeten the poison.
    • As seen from Cromwell’s beheading, due to his role in the overly ambitious reform and uprooting of Roman Catholicism.
  • When you destroy the familiar, you create a void that people will fill. The past is powerful. Borrow from its legitimacy to create a comforting and familiar presence. They will sustain your actions with a familiar and romantic feel, cloaking the nature of the changes you are attempting.
    • Because the past is buried and dead, it gives you the flexibility to interpret them as you want.
    • The Roman used this when transforming monarchy to republic
    • Romans used this device when they transformed their monarchy into a republic. They may have installed two consuls in place of the king, but since the king had been served by twelve lictors, they retained the same number to serve under the consuls. The king had personally performed an annual sacrifice, in a great spectacle that stirred the public; the republic retained this practice, only transferring it to a special “chief of the ceremony, whom they called the King of the sacrifice.” These and similar gestures satisfied the people and kept them from clamoring for the monarchy’s return.
    • When Romans declared Christianity to be Rome’s official imperial religion, the evocation of light and fertility (Saturnalia, Festival of Lights, Germanic celebrations of the re-birth of the Sun which all happened in December) could not be ignored. In 354 AD, the Christian church co-opted the birthday of Mithras (theAryan god of light), and declared December 25 to be the birthday of Jesus Christ.
  • The ones that finish a revolution are rarely the bones who started it.

How you Show Up

Be flexible, assume formlessness

  • Just like an animal’s shell protects them from dangers, they also make them slower and inflexible to change. Same can be said for governments and systems that create protection that ultimately make them rigid and vulnerable to change.
    • Rigidity heightens the desire for change.
  • In a Chinese tale, a farmer observed a hare running straight into a tree, breaking it’s neck. Afterwards, he waited in front of a tree waiting for the same thing to happen. He never caught another hare, and was ridiculed. Same could be said for someone who tries to govern using the same strategies as those of early kings. They would be doing the same thing.

Presence dilution

  • “Love never dies of starvation, but often of indigestion” - Ninon de Lenclos
  • “Absence diminishes small passions, and inflames greater loves. Just as wind douses a candle, and fans a fire.” - François de La Rochefoucauld

Reputation and Respect

  • If your reputation is low, then there is not not much to lose, but your opponent does have to lose. And eve. If they attack you, they won’t be able to do much, becAuse there is nothing to attack. Do you can go more all in. Reputation is critical though, so make sure this doesn’t backfire.
  • Be the master of your fate, and your reputation.
  • “When the great lord passes, the peasants bow deeply and silently fart” - Ethiopian proverb

Symbology

  • A symbol is a powerful form of expression that summaries several concepts
    • For example, the Sun King symbology created for Louis XIV of the Sun King, symbolizing his provider god like status.
    • Or Diana the mistress of king Henri II of France, who dodged the common fate shared by many other elder mistresses of abandonment, by scattering around the reign multiple symbols related to her (Diana’s favorite colours [black and white] imprinted in several buildings and images, associations with goddess of hunt Diana), and creating a motif intertwining her initials with his, that king Henri disseminated widely. There was no escaping her presence. Until his death, in 1559, he remained faithful to her.
  • A symbol is a shortcut of expression, containing in itself several dozen meanings. Symbols are generally subtler and less threatening than words, which can be too incisive.
  • Primacy of sight amongst the senses: “Truth is generally seen, rarely heard” - Baltasar Gracián
]]> 2024-12-08T00:00:00+00:00 Lessons from Roman History https://lopespm.com/notes/2024/10/13/lessons_roman_history.html https://lopespm.com/notes/2024/10/13/lessons_roman_history.html Keeping up with the theme on capturing lessons from content I consume, I’ve recently seen an interview from Lex Fridman to Gregory Aldrete, an historian specializing in ancient Rome and military history, in which Gregory eloquently spoke about many of the fascinating details on the rise and fall of ancient rome. Here are my takeaways from it.1

Rise and Fall of Rome

Romans were obsessed with the past

Romans were absolutely obsessed with the past, especially with their own family.

Entering an aristocrat Roman’s house, the first thing you would see would be a big wooden cabinet with several rows of wax death masks. These masks were imprints of Roman aristocrats at the time of their death.

Every child in the family had obsessively memorized every accomplishment of every one of those ancestors: their career, what offices they held, what battles they fought in, what they did.

At a funeral, people would talk about all the things their ancestors had done. The children would take out these masks, tie them onto their own faces, and wear them in the funeral procession. They were wearing the face of their own ancestors. You, as an individual, weren’t important. You were just the latest iteration of that family, and was a huge weight to live up to the deeds of their ancestors.

Brutus honors the past, kills Julius Caesar, Roman Republic dawns

Rome started out as a monarchy. They had kings and were not happy with their kings. Around 500 BC, they held a revolution and they kicked out the kings, and the Roman Republic started at that point. One of the people who played a key role in this was a man named Lucius Junius Brutus.

500 years later, Julius Caesar2 came along as the culmination of a sequence of generals trying to overtake Rome and declare themselves as kings. Even though he was populist who provided entertainment to the state, Julius Caesar was arrogant, didn’t hide his power, ignored the senate, and got several people angry. Romans don’t like kings.

Just so happens that one of Caesar’s best friends is Marcus Junius Brutus. People went to Brutus’ house and wrote graffiti saying “Remember your ancestor” or “You are no real Brutus”. He had no choice. He forms a conspiracy, and on the Ides of March, 44 BC, he and 23 other senators take daggers, stick them in Julius Caesar, and kill him for acting like a king.

Brutus killed his best friend because of something his late ancestor did.

Filling Julius Caesar’s power void

Julius Caesar and left a power void. There are many contenders for filling it up:

  • Mark Antony: a relative and supporter of Julius Caesar. Most expected him to be the new Ceaser.
  • Lepidus: another of Caesar’s lieutenants.
  • Senate: which wants to reassert its power, to become the dominant force in Rome again.
  • Assassins who killed Caesar: Led by Brutus and Cassius.
  • Pompey’s son: Pompey was Caesar’s great rival. Pompey’s son, Sextus Pompey, was at the time a warlord who seized control of Sicily, one of the richest provinces, and amassed a whole navy.
  • Octavian: Julius Caesar’s 18-year-old kid grand nephew. When Caesar’s will was opened after his death, he posthumously adopted Octavian as his son.

Octavian emerges victorious

By now being Caesar’s son, Octavian gets to rename himself Gaius Julius Caesar Octavianus, and it just so happened that in the Mediterranean, there were about 12 legions full of hardened soldiers following the orders of a man named Gaius Julius Caesar.

As such, Octavian inherits an army overnight and becomes a player in this game for power, and a civil war starts.

Octavian emerges from it as the victor. He wasn’t a good general and lost almost every battle, but was politically savvy and very good at manipulating public image and propaganda. Octavian waged a propaganda war against Antony, portraying Antony as a foreign aggressor allied with an enemy queen, Cleopatra, and who was an official enemy of the Roman state). Octavian takes what’s a civil war and makes it look like a war against a foreign enemy.

Octavian is the “king”, but can’t act like one

Octavian now becomes the sole ruler, essentially a king, but could not take the same approach as Julius Caesar, otherwise he would end up murdered all the same.

Instead, he was very modest, lived in an ordinary house like other aristocrats, wore just a plain toga, was respectful to the Senate, and ate simple foods. He’s someone who cared about the reality of power, not its external trappings. He wanted real power, not the appearance of it.

In terms of government, everything seemed the same from the outside, but in reality Octavian was able to retain absolute power. He did by resigning from all public offices to give that appearance, but at the same time got voted to have the powers of a consul, by which he could command armies, he got tribune power, to control meetings at the Senate, he could veto anything, and got several other powers.

Each year elections are held, and notionally, these people are in charge. But floating off to the side, there Octavian, who can just appear and say that “I don’t like this, let’s change it”

Nor can he be named like a king

Octavian wondered what to call himself. He couldn’t call himself a king, or anything that could suggest it, so instead he picked ambiguous names, that when joined an interpret in a certain way, would proved to be powerful:

  • Augustus: Augustus could mean that someone is very pious, or could mean that something is divine.
  • Princep3: Meaning, first citizen. Could be interpreted as a citizen just like everybody, or the first citizen, superior to all the others.
  • Imperator: traditionally something that soldiers shout at a victorious general who’s won a battle. Octavian took this as a permanent title, implying he was a good general4.

What being Roman meant

It’s wonderful to contemplate how the roman empire in about 100 AD overlaps with the regions where olives could be grown. Romans consumed olives, grapes, wheat. Barbarians meat, dairy, beer. When you are a farmer, you tend to stay in the same place, when you raise cattle you follow them around. They are two fundamental forms of living. Diet was a big part of their culture and one of characteristics that was considered fundamentally Roman. Not having their diet was barbaric.

Roman empire crumbles

There are many factors that could explain the fall of Rome, and there is not a single clear cut explanation. Even the date of the empire fall is debated.

Geography, climate, religion, disease (there were a whole series of waves of plague that started to hit under Marcus Aurelius and continued after him, which seemed to caused real serious death and economic disruption), Marcus Aurelius leaving is succession to his child (who turned out to be deranged), instead of picking the best suited person for that role 5, as it was done on the previous 80 years, which is often regarded as the high point of the Roman Empire.

Power of the past

No way to escape the power of the past

Gregory and his wife wife wrote The Long Shadow of Antiquity: What Have the Greeks and Romans Done for Us?”, where they provide several examples of things that we think are just in truly unique parts of our culture or things that we think are just innate to human nature, that are actually rooted in the past, such as government, education, art, architecture, language / words, culture, medicine, habits, law, the way we get married, the calendar (Julius Caesar was the one who basically came up with the 365 days, 12 months, leap years).

We’re the accumulation of the knowledge of several generations that have come before us. Everything we do is based on that. Otherwise, we’d all just be starting at ground zero.

Understanding the past to mold the future

It’s vital to have some understanding of the past in order to make competent decisions in the present. Not just in your own life, but it’s in understanding others. You need to understand where they’re coming from, where they came from, and what shaped them, and what forces affect them.

The study of history is the best medicine for a sick mind; for in history you have a record of the infinite variety of human experience plainly set out for all to see: and in that record you can find for yourself and your country both examples and warnings: fine things to take as models, base things, rotten through and through, to avoid.

― Livy, The History of Rome

People from antiquity had different environments, technologies, and information available, but were just as sharp as we are nowadays. They were not stupid. Even though it might seem that many concepts and ideas were invented by our contemporaries, many lessons, successes and mistakes were already discovered in the past 6. The real challenge is to incorporate them into our own lives.

  1. Gregory Aldrete portrayed several of these concepts beautifully, so some passages of this article are literal paraphrasing of his discourse.↩

  2. Kaiser, Tsar, Tzar, Czar. These are based on the word Caesar.↩

  3. Princep is the reason why we have “Princes” and “Princesses” afterwards. Everyone wanted to be like Octavian.↩

  4. It’s from the imperator that we get the word emperor and empire.↩

  5. How did these emperors pick the best suited person for that role, while still sticking with the tradition of leaving succession to the emperor’s children? By adopting middle aged men that they considered fit for the role.↩

  6. For example, Cicero (assassinated on the orders of Octavian and Marc Antony) is considered one of the prime examples of a good orator, and wrote at length about it. Many of his experiences, skills and tricks are still used nowadays by several orators.↩

]]> 2024-10-13T00:00:00+00:00 Connection with reality https://lopespm.com/notes/2024/09/15/connection_with_reality.html https://lopespm.com/notes/2024/09/15/connection_with_reality.html 。


As we experience an increase of massive amounts of AI generated content, this note will go through the role and added value that a connection with reality and nature brings not only to your business and craft, but also to yourself.

Striking a balance

Walking or hiking has been a recurrent weekly habit of mine for several years, and its one of my favorite activities.

I find it essential to have a walk outside the office / home and be in contact with the surrounding world, be it in nature or around the city. This is especially relevant for those, like me, who have their daily jobs working at the computer.

I love to be enthralled in the digital world, think about abstract concepts, and use digital devices to, for example, communicate this content to you. But like anything else, a balance needs to be kept, and a balance between the digital and real domains seems essential.

The added value of (connecting with) reality

As generative models get better, more and more people will just assume that a given piece of text, image or video is likely fake, and I believe there will be an increased demand for a connection with the real world through experiences such as in-person events, travel, spirituality, or consumption of digital content by certified entities.

Businesses and creators that prove their connection with the real world and real people, will likely have an edge and rise against the noise.

So if you’re creating a business, a piece of content, anything really, think how it relates with the real world. Ask yourself: how can you provide that connection? How can you provide reality as a service?

Keep yourself connected with reality

Apart from all this, and just as important, it’s for you to maintain your contact with reality.

So if you feel you spend too much time around the internet, digital devices, TV series, etc, I invite you to the outdoors, the world outside. It’s free, it’s accessible. You just need to open the door and step out, provided you are fortunate enough to have the required physical capabilities. I’m pretty sure that it will make your life much better.

]]> 2024-09-15T00:00:00+00:00 Interview Learnings from former CIA Intelligence Officer https://lopespm.com/notes/2024/09/12/interview_learnings_cia.html https://lopespm.com/notes/2024/09/12/interview_learnings_cia.html 。


I’ve been getting into the habit of writing down what I learned from content I consume and I’ve recently seen an interview with Andrew Bustamante, a former CIA intelligence officer on the Diary of a CEO with Steven Bartlett.

Andrew Bustamante is a former covert CIA intelligence officer, US Air Force combat veteran, and Fortune 10 corporate advisor, who founded Everyday Spy, where individuals and teams are trained to leverage influence, intelligence and intent. Techniques once reserved for elite spy agencies can now serve everyday people in their pursuit of personal and professional objectives.

Andrew shared his candid and unfiltered views on the human condition and how his previous CIA intelligence officer (commonly known as a spy) can be leveraged towards gaining an advantage in business and everyday life, and in this article I will share some of the highlights and learnings from this interview. Here are my key takeaways:

Do an Action, Any Action

For you to be a step ahead of everyone else just do an action, any action, and you’ll already be ahead of everyone. Even if you fall, you’ll already be four steps ahead of everyone else.

Two kinds of people

There are two kinds of people:

  1. Those who fall victim to their fears and will be stuck on a cycle of consumption, he calls them the cogs, the bobbleheads. He mentions that we actually need them, because they are needed by the kind.
  2. Those who face their fears and produce the things that the cogs want, but fear of doing. So they exploit them.

Motivation & Manipulation

Motivation and manipulation are two sides of the same coin.

Using motivation just exploits something that people are already prone to do, and we just give them something that they want.

If they’re not motivated, then we use manipulation, so they can actually give us what we want.

Moral Flexibility

Moral Flexibility is a gateway for manipulation, which makes something acceptable in one context, but unacceptable in another context.

Public, Private and Secret Spheres

Then there are three spheres: the public, the private and the secret.

Public Sphere

The public sphere is the safe zone:

  • It’s what you wear
  • It’s what you say in Instagram
  • It’s what you state to everyone
  • It’s the identity that you want other people to perceive

Private Sphere

The private sphere is what the people in your inner circle see:

  • They know your birthday
  • They know your favorite ice cream
  • They are the ones that really know you

So it’s small by definition. This makes you feel good and special, and you have this elite group which is distinct from the public.

Secret Sphere

The secret sphere is where all of your secrets lie:

  • It’s the affair you’re having
  • It was the molestation you had as a kid
  • It’s all of your dark thoughts

To move from one person’s public to secret sphere, you need to pass by the private one first.

We want to have someone to tell our secrets, but we don’t have enough trust on the people that are in our private sphere.

Once you get to someone’s secrets, they’ll trust you so much that even if you break their heart. They’ll resist leaving you, because it is very rare to find someone who we can tell our secrets to.

4 Core Motivations

About the four core motivations, they are described by RICE: Reward, Ideology, Coercion and Ego

  1. Ideology is the biggest driver. This could be politics, values, morals, etc
  2. Ego comes next. Even showing yourself to the world for people to see and to validate how you want to be seen is ego. Andrew mentioned that Madre Teresa wanted to be seen as someone who is a martyr.
  3. Reward: Money, sex, alcohol, drugs, etc
  4. Coercion: this is the negative one, and includes things like blackmailing

2 Questions, 1 Validation

If you want to get information from a person, then you can ask two questions and then follow up with the validation.

For example, if the other person mentioned that they have an issue with their wife, then you ask:

  • “Oh what happened?”
  • After their reply you ask: “Did she really do that?”
  • Again, after their reply, you do a validation, so it doesn’t sound like an interrogatory, by saying for example “Yeah, I had a girlfriend that did the same thing”
  • After their reply, you then ask another two questions, and so on and so forth.

Doors and Windows

Use windows instead of doors when leading a conversation.

Opening a door is when you change the topic completely. For example, if the conversation is about the weather and all of a sudden you ask about someone’s income. The analogy is that you’re breaking through someone’s door.

Instead you’ll want to open a window, which is to leave a hint for the other party that will organically lead them towards a new ramification of the conversation. You open a window by giving a queue, a suggestion, or an implicit conversation seed.

Volunteered Information

People will also volunteer information to you, and you can actually assess that by the information that they give you through their statements.

The Power of Questions

The person that is asking the questions has the power over the conversation, and not the person who is saying the most words. For example, in the interview it was agreed by both that Steve was the interviewer, so Steve had the power there.

SADRAT in Business

Andrew mentioned how the SADRAT methodology can be used in business. SADRAT stands for: spotting, assessing, development, recruiting, agent handling and termination.

  • Spotting is finding the client, this is, you need to kiss a lot of frogs to get to the prince.
  • Recruit means you provide a given product in exchange for their money.
  • Assess is about finding if someone is going to be a good productive client, so you can place your Investments on the clients that give you the return on the time span that you want. This can be known through their common traits, their profile, etc.

Detecting Lies

In order to know if someone is a liar, you need to first establish a baseline. Once that is acquired, you need to put pressure on them to see if they react differently.

The difference between good liars and bad liars is that bad liars tend to twitch and turn, as if they were in a hot seat when they get exposed to pressure.

Good liars, on the other hand, they occlude all of these signals. Good liars tend to be the ones at parties that ask a lot of questions, and you felt like they were so friendly, but in reality you know nothing about them.

Perception vs Perspective

Perception is your gut view. Perspective is seeing the event through other lenses.

You should not trust your perception, because most of the time it is wrong and it is emotion based. You need to use your rational side to process challenging situations.

To achieve that, you need to be inoculated by getting exposed to small doses of something you fear. For example, asking someone if you’re overweight. You’ll get the physical fear arousal, but once it’s over no real harm was done.

You can train yourself to not respond to that physical jolt to make your emotional part slower, and your rational part faster.

]]> 2024-09-12T00:00:00+00:00 Obsidian Encrypted Backup - Google Drive, Dropbox, OneDrive https://lopespm.com/notes/2024/09/11/obsidian-backup.html https://lopespm.com/notes/2024/09/11/obsidian-backup.html 。

After several years using Evernote, I’ve eventually migrated all my notes into Obsidian, which allowed me to have full control of my notes, in a format that I could use, move, or leverage upon. As a consequence, my notes would no longer live in the cloud and my personal device, so any redundancy and backups would need to be guaranteed by me, via my personal periodic backups.

Having more than a decade’s worth of notes relying on a single redundancy made me somewhat uneasy, so the options I could think of were either:

  1. Subscribe to Obsidian’s Sync service, which would recurrently cost $4 every month, every month. My encrypted notes would be tethered to Obsidian’s cloud service
  2. Have a custom solution that leverage’s Obsidian’s outstanding customizability, compress and encrypt all my notes, and use a cloud service to host this archive. I would have the flexibility to choose any cloud provider I would desire.

I’ve chosen option 2., using the Google Drive cloud service, and in this note will share how you can too.

How to do it

The idea is simple: use the obsidian-shellcommands shell plugin to run a custom script, whenever Obsidian quits (this event is configurable, but I find the application).

Step 1. Custom script that encrypts and backs up all notes

First, save the below script into a folder in your computer (for example, at /Users/yourunixname/backups/my_backup_script.sh), and update it with your own Obsidian, backup destination folders and your own archive password:

#!/bin/zsh

obsidian_notes_folder="<your_obsidian_folder>" ; # For example, /Users/yourusername/Library/Application Support/obsidian
obsidian_notes_tar_archive="${obsidian_notes_folder}/obsidian_backup.tar.gz" ;
backup_folder="<folder_where_the_final_encrypted_backup_will_be_placed>"; # For example, /Users/yourusername/Library/CloudStorage/GoogleDrive/MyDrive/backup_folder

echo "Starting to compress obsidian notes..." ;

# Create a .tar archive that contains all the contents inside the obsidian folder
tar -czf ${obsidian_notes_tar_archive} ${obsidian_notes_folder}/obsidian_backup

# Compress the .tar archive into an encrypted .7z with password "PasswordOfYourChoosing"
# In this example, 7za installed from the nix package manager is used
/Users/yourusername/.nix-profile/bin/7za a -tzip -mem=AES256 -mx=0 -mmt=12 -pPasswordOfYourChoosing ${obsidian_notes_folder}/obsidian_backup.7z ${obsidian_notes_tar_archive} ;

# Move the .7z file into the the backup folder (e.g. your Google Drive / Dropbox / OneDrive folder)
mv ${obsidian_notes_folder}/obsidian_pedrolopes_nodes_backup.7z ${backup_folder}/obsidian_backup.7z ;

echo "Finished compressing and moving to backup folder"

The comments are mostly self-explanatory, but essentially this is what the script does:

  1. First create a .tar archive that contains all the contents inside the obsidian folder
  2. Compresses the .tar archive into a password encrypted .7z file. Remember to update the password with your own private password
  3. Moves the .7z file into the destination folder, which could be the folder used by your cloud storage sync folder of choice (e.g. your Google Drive / Dropbox / OneDrive folder).

Since the final file name on 3. is always the same, it will be re-written, but likely your cloud storage sync will keep track of the different versions, as they change, which could progressively inflate your quota usage. If that is a problem, just purge them periodically using your cloud storage interface.

You can test drive your script by granting execution privileges to your script (chmod +x <script_file_name>), and running ./<your_script_name>. Your final encrypted archive should appear on the final backup_folder

Step 2. Run the script when Obsidian quits

Now that we have the script ready, it makes our life easier if it is run automatically, upon a given Obsidian event, such as when Obsidian quits. Running a shell script upon a given Obsidian event is made easy by using the obsidian-shellcommands. These are the steps to set it up:

1. Go to Obsidian -> Settings. Then select the “Community Plugins” option.


1.1. On the “Community Plugins” option, click “Browse”. There, search for “Shell”. The one you want to install is Shell Commands by Jarkko Linnanvirta


2. Now that the plugin is installed, go again to Obsidian -> Settings. You should see in the bottom left, under the “Community plugins” pane, an option named “Shell commands”. Click it.


3.1. On the “Shell commands” plugin, select the “Shell commands” tab, and inside it, click “New shell command”, and the created row, click its respective cog icon. This will show you a new modal with several tabs.

3.2. On this command modal, select the “Environments” tab and insert on the “Default shell command” something like this: zsh /Users/yourunixname/backups/my_backup_script.sh. This assumes that your script is located at /Users/yourunixname/backups/my_backup_script.sh


3.3. Still on this command modal, select the “Events” tab. There you can choose when should the script execution be done. For example, search for “Obsidian quits”, and enable it (there is a toggle in the right)


All done! After performing these steps, your backup script will run whenever you chose to, and the encrypted backup will be created and placed in the location that you chose on Step 1. above 🎉

]]> 2024-09-11T00:00:00+00:00 Investment Tracker - Free Spreadsheet Template https://lopespm.com/notes/2024/09/01/investment-tracker.html https://lopespm.com/notes/2024/09/01/investment-tracker.html 。

Do you know:

  • Your current net worth?
  • How much money do you have in the bank?
  • Which are your most profitable investments?
  • Which investments do you currently have?
  • How much of your revenue (e.g. salary) contributes to your wealth?
  • How did the above evolve over time?

If coming up with the above answers is hard, or their values are fuzzy, then it might be time to revamp your investment tracking.

This article will:

  • share a Google Spreadsheet template to track your investments, and explain it
  • provide a guide on how to use it
  • suggest a simple process to make sure the above questions will not be a problem anymore

The Template

Get it

How it is structured

The template has several sheets, which are built to have the essential tracking elements that can be immediately used, while also allowing for extensibility for your own custom case.

Dates sheet

The entire sheeet revolves around these dates. Each of these represents the temporal snapshot of all your assets. All the other sheets use these dates as the starting point for their values, and this would be first thing you would add whenever entering a new series of entries

Revenue sheet

Represents all the resources coming in, which are not a result of your investments. This could be the income from your salary, side-hustle, the app you are selling on the app store, etc.

Account sheet

There are two example account sheets provided with the template, each representing a different account on your portfolio, but you can add how many you would need. These could be your bank account, transaction broker, private pension, etc. To create a new one, grab one of the example account sheets, and copy it into a new spreadsheet.

On the template, the Example Account1 shows how a bank account could roughly look like, and Example Account2 how an investment platform (such as Vanguard) could look like.

Each of the account sheets is structured in the following way:

  • Current status: a look at the present, and comparison with the last update
    • Date Column: referenced from the dates sheet, representing the date of this current snapshot
    • Top up movement: any money that you deposited in this account since the last date, would be accounted for here. For example, salary that is deposited, on deposits made in an investment broker. The objective here is to keep track of the money coming into a specific account, so that its performance doesnt look better than it really is.
    • Not Invested: all uninvested money sitting in the account. For example, money in a bank account which does not yield interest.
    • Invested: all money invested in this account. For example, the amount of money invested in a transaction broker, which is currently invested in stocks or ETFs.
    • Investment Delta: the investment difference between the current date and the last date.
    • Investment Delta, minus topup: the investment difference between the current date and the last date, minus the top up. This value attempts to represent how much money was actually generated by the investment since the last date.
    • Total: the total invested and not invested amount in the account.
    • Total Delta: total difference between current date and last date.
  • Expected gains: predictions of investment gains
    • Expected Yearly Growth % (Gross): for example, if overall the invested money is in a Fixed Term Deposit, this value would represent the gross yearly interest accrued from it, pre-taxes.
    • Expected Yearly Growth % (Net): this value represents the actual expected gains from this investment. For example, taking the Fixed Term Deposit example above, this would be the expected yearly interest, after taxes.
    • Expected gains 6 months / 1 year / etc: taking into consideration the Expected Yearly Growth % (Net), which would accrue gains from the investment over these time spans.

Summary sheet

This is where it all comes together. All the accounts are summed up, resulting in these new relevant columns:

  • Total: the total across all investments, for a given date
  • Total Delta: total difference between current date and last date.
  • Total Delta minus Revenue (i.e. profit / loss): the aim of this column is to roughly represent how much of your net worth is being eaten up by your expenses:
    • If this value is negative, it means that either your costs are too high, your investments are not yielding yet high values, or both.
    • If this value is neutral, it means that your overall investments are completely covering your costs.
    • If this value is positive, it means that your overall investments are not only covering your costs, but are also increasing your net worth on top.

How to use it

Set up

  1. Copy the template, by opening it and clicking on File -> Make a Copy
  2. Go to the dates sheet, and leave only the top two dates. The first date would be the previous month, and the second would be the current month. Delete the rest.
  3. Go to the other sheets, and delete all the rows which don’t have a date now
  4. Zero out the first line on each of the account sheets for Top up Investment, Not Invested and Invested.
  5. Zero out the first line of the revenue sheet; or add your revenue from that month.
  6. Use the existing example accounts to host your own accounts. If a new account is needed:
    • Copy one of the account sheets, by right clicking on the sheet’s tab, and selecting Duplicate
    • Add a new column to the summary sheets, representing the new account. Change its formula’s value to get the information from your new sheet. You can do this by selecting the cell and changing the name of the referenced sheet, to the name of your new sheet. For example, from 'Example Account 2'!G2 to 'Monzo Bank Account'!G2.
    • Still in the summary sheets, double click the chart, select the Setup tab, and add the range of the new column to the list. For example, if the new column was on the D column, then the data range would change from H7,A2:A1000,B1:B1000,C1:C1000,G1:G1000 to H7,A2:1000,B1:B1000,C1:C1000,D1:D1000,G1:G1000. Then, on the Series section of the Setup tab, click on Add Series and add the one from your new account.

Recurrent workflow

Once you have the above set up, I suggest that on the first days of every month, right after your main source of income is deposited (e.g. salary), that the sheet with the current values of each of the accounts. This is key. If this update is not made periodically, eventually there will be gaps in the tracking, and its data will lose relevance. The process would look something like this:

  1. Add the current date to the dates sheet
  2. On each the the accounts, expand the date to the new one, and fill in the current Top up Investment, Not Invested and Invested values
  3. All done!

Final thoughts

Feel free to expand and customize the above tracker to your needs, and let me know how it could be improved and which was its impact on your investment tracking. Would love to hear your thoughts about it.

]]> 2024-09-01T00:00:00+00:00 UK Investment Guide https://lopespm.com/notes/2024/08/26/uk-investment-guide.html https://lopespm.com/notes/2024/08/26/uk-investment-guide.html ，同时不要出现编号。

The sooner investments are made, the more profound their results will be, and if you are living in the United Kingdom and not taking advantage of its welcoming investment opportunities to put your money to work for you, you’re missing out. Don’t postpone it.

This article provides concise, practical, straight to point suggestions on how to kick start your investment journey.

Disclaimer

  • I am not a qualified licensed investment advisor. The contents in this article are not financial advice.
  • I will not and cannot be held liable for any actions you take as a result of anything you read here.
  • Always do your own research and seek independent financial advice when required.
  • When investing, your capital is at risk. Investments can rise and fall and you may get back less than you invested. Past performance is no guarantee of future results. Other fees may apply.

Foundation

Why

Growing wealth has been vilified and correlated with the unscrupulous and greedy. Far from that, money is more than a number and currency: it provides options. The option to say no to that task / job you don’t want to do; options to provide for your family and close ones; options to give back to society and the world; options to support your health, well being and comfort. There is nothing wrong with that.

So why would you consider investing your hard earned money? Why do it as early as possible?

Let’s say you are 20 years old, and have zero net worth. No money in the bank, no assets. After that point, every month, for the next 30 years, you invest £250 every month (roughly equating to £8.4 per day). Let’s assume that investment yields a yearly 7% return on investment every year, on average (which is viable assumption, considering the SP500 historical performance). By 50 years old, this would be the result:

  • Contributions: £90,000 (250 * 12 * 30)
  • Interest earned: £193,382 (thanks to compound interest you will gain this much. Einstein called compound percentages the 8th wonder of the world)
  • Total: £283,382

Imagine how much higher these values could get if the invested values are increased. You can use this calculator to have an idea.

Long term game

Forget about get rich quick schemes. Going down that path would be comparable to buying lottery ticket and wish for a winning prize, but the odds of getting struck by lightning are nearly 20,000 times higher than hitting the winning numbers for the next Powerball jackpot.

Building wealth is a long term game which requires discipline, structure and time.

Methodology

Create an emergency fund

Make sure you have the equivalent of about 6 liquid months worth of your income available. Life is unpredictable, so make sure you have a reachable safety net which you can rely on and use promptly.

Pay off your (consumer) debts

Compound investment works both ways: the more you invest, the more your wealth “snowball” will pick up along the way, growing exponentially. On the other hand, if you have debts that require you to pay interest, the inverse applies, progressively eating into your wealth.

As such, gather all your debts, order them by their interest rate, and make sure to pay the ones with the highest interest rate first. Prioritize non-equity based debts, such as credit card debts. Mortgages are equity based, so a portion of your payments goes towards your equity, so they would have less priority compared with pure consumer debt, but make sure they will not overburden you.

Budget

Budgeting doesn’t mean living in extreme frugalness. Budgeting is about knowing what you need and want, and how much of your income would you be willing to allocate to them. Budgeting makes it easier to reason about your expenses, and having control over them, rather than them controlling you.

Make it as simple or complicated as you would want. You can start by putting in a sheet your monthly income, and subtract from it your fixed (e.g. rent, mortgage, internet bill) and variable expenses (e.g. entertainment, electricity bill, travel). In the end, the objective is that you have a better grasp of how much comes in / goes out. This will inform you on how much disposable income you will have available towards investments.

Automate

Unless you are a professional trader, you don’t want to spend the entirety of your (limited) time and energy maintaining your investments. If so, that becomes a chore / job by itself. The whole point is to make money work for you, not the contrary.

So automate everything as much as possible. What you can’t automate, make it into a well defined process that requires minimal effort.

I suggested you review / maintain your investments only once per month, every month. If you are an employee, make this shortly after your paycheck comes in.

Systematize. Know where your assets are. Visit them.

  • Every month, invest as much as possible from your income (this amount would be informed by your budget, and your risk appetite). You can also automate this process by scheduling a periodic amount from your bank account into your investment platform. I don’t particularly like this full automation strategy, since I prefer to adapt my investment amount to what I predict my spending will be on the following month, allowing for a tighter / more aggressive investment strategy. I personally prefer to have as little uninvested money as possible. YMMV.
  • Make a spreadsheet. Doesn’t need to be fancy, just functional. It will only be used by you (in principle), so organize in however way you think it is best. Personally, I have different pages for each bank, trading platform, p2p platform, etc. Each of these pages has an history of the current invested amount in these platform, per month. Then an extra page summarizing / summing up all of these assets, with some optional graphs. Having this visual historical summary of all your assets in a single place is gratifying. Seeing how far you came on your investment journey, explicit in a graph, is incredibly fulfilling and powerful.
  • Update this spreadsheet periodically, and re-balance your assets as needed. I recommend every month, right after you receive your salary. There are ways to do it automatically, but taking this time to go through all your assets provides visibility on any gaps and strategy changes that need to be done. Personally, this takes me one hour. That’s how long it takes to perform maintenance on my investments: 1 hour every month, that’s it.

Diversify. Adapt to your situation

Having your investments diversified across different baskets allows you to balance your risks are diffuse your liabilities across different spaces, which ultimately protects you.

The strategy you would want to adopt for your investments in your 20s, likely won’t be the same as the strategy in your 60s. For example, later on in life, you might want to invest in safer investment vehicles, such as UK Government bonds (essentially, the country would need to go bankrupt for you to lose your money), savings accounts, cash ISAs and real estate.

Investment Vehicles

With our foundations in place, it’s time to dive into the nuts and bolts.

Max out your workplace pension

Usually, if your are employed, your employer will make regular payments into your pension pot on your behalf. And, depending on the plan you belong to, you may also make monthly contributions which will be taken from your salary. Some employers increase their contributions if you increase the monthly amount you put aside as well. This tends to be a direct match – if you put aside an extra 1%, they add an extra 1%, but there will be a maximum level they will match.

There are two allowances you should keep in mind:

  • Your annual allowance is the maximum level of total pension contributions to all of your pensions (made by yourself, your employer(s) or anyone else) that will receive tax relief in a tax year. It is currently £60,000, although could be lower if you have an adjusted income over £240,000 or you have already made a taxable withdrawal from any of your pension plans.
  • The lump sum allowance (LSA) is the maximum amount of tax-free cash you can take from your pension savings in your lifetime. You can take 25% of your pot tax-free, as long as this amount is not higher than the LSA.

Own the World: Stocks, ETFs

Google, Apple, Tesla, NVIDIA: what if you could own a part of them and profit from their earnings by receiving dividends from them and / or getting capital gains from their (hopefully) raising share prices?

Turns out, you can. Either by buying their individual stocks, or better yet, you can invest on a index fund, such as the S&P 500.

S&P 500

S&P 500 contains the top 500 U.S.A. publicly traded companies with a primary emphasis on market capitalization. Most interestingly, this list isn’t manually curated by a human or an opaque heuristic. The list is based on the market capitalization only, and it is considered one of the best gauges of large U.S. stocks and even the entire equities market because of its depth and diversity. This dilutes the risk you are exposed to, and historically has provided a realistic yearly ~7.5% return on investment.

You can’t invest directly in the S&P 500 because it’s an index but you can invest in one of the many funds that use it as a benchmark and track its composition and performance.

To make it easier, you can get a S&P 500 Exchange-traded fund (ETF). ETFs and index funds are similar in many ways but ETFs are considered to be more convenient to enter or exit. They can be traded more easily than index funds and traditional mutual funds, similar to how common stocks are traded on a stock exchange.

Here are two S&P 500 ETFs, which have a low maintenance cost:

  • S&P 500 UCITS ETF - Distributing (VUSA): the dividends are paid out to you, and you can choose to reinvest them, invest them elsewhere, or just cash them out.
  • S&P 500 UCITS ETF - Accumulating (VUAG): the dividends are automatically reinvested into the ETF, which makes the process more automatic, in case you want to reinvest your earnings (which likely, you should). Note that, even though these dividends are reinvested directly into the ETF, they are still considered as earnings by the HMRC, so in the tax perspective, there is no difference between distributing and accumulating.

Other ETFs

Investing on a S&P 500 is already a significant step up from plain low interest savings accounts, but you would be overly exposed to the U.S.A. market. As such, you can consider ETFs which further diversify your exposure, such as the FTSE All-World UCITS ETF - (USD) Accumulating (VWRP), which is comprised of large and mid-sized company stocks in developed and emerging markets.

The above ETFs are provided by Vanguard (which tends to have low maintenance costs for their ETFs), and can be bought in several trading platforms, but there are many other reputable companies providing similar ETF, such as BlackRock.

Where to buy these Stocks and ETFs

  • justetf: is a great website to explore and get details on costs and performance of an ETF you are aiming to invest in
  • Vanguard: if you are starting out on your ETF investment journey, I strongly suggest this platform to invest in your ETFs. Making deposits and managing investments is straightforward and has the right amount of information and process, in order to not be overwhelming
  • Interactive Brokers: this platform has been around for a long time, is low cost, well-capitalized, provides for very low cost exchange rates, and allows you to invest in a large amount of available stocks and ETFs. The trade-off is that it takes longer to onboard, due to its interface complexity and wide range of tools and information.
  • Trading212: commission-free platform that doesn’t apply any commission on trades, but does charge 0.15% on currency conversion fees. Make sure to take these hidden costs into account.

Vanguard, Interactive Brokers and Trading 212 are regulated in the UK by the FCA. I recommend you spread out your investments in different platforms, in order to reduce the risk / hassle in case any of them is shut down or rendered illiquid.

ISA

Every time you sell the above ETFs or Stocks at a gain (for example, you bought £100 worth of an ETF, and then sold it for £150), and every time dividends are received (regardless if they are paid directly to you, or reinvested), you will need to pay taxes on them, unless you are still within their respective allowance. 1

One way to not pay any of these taxes2, is to use a stocks and shares ISA. The stocks and shares ISA is one of the types of ISA. Other ones being cash ISA (type of savings account that offers tax-free interest), innovative finance ISA (peer to peer lending with tax-free interest) and Lifetime ISA (tax-free savings account that gives a 25% bonus if used for retirement or to buy a first home).

As of the time of this writing, the maximum yearly amount allowed to be saved via ISAs is £20,000. This means that across any of the above 4 types of ISAs you can only deposit in them up to £20,000. You can withdraw them at any point in time. 3

Some specifics on this:

  • Creating a Stocks and Shares ISA is fairly straightforward. The above brokers (Vanguard, Interactive Brokers and Trading212), all allow for you to create a Stocks and Shares ISA account. This account is exactly the same as a normal / general account, except that it is tax sheltered. The ISA account would be separate from a general investment account you might want to create, in case you already reached your £20,000 limit, and want to continue investing (which I recommend).
  • You can create ISA accounts in multiple providers of any types you choose, but remember there is a global £20,000 limit. For example, if you have a Cash ISA on HSBC where you deposited £15,000, and Stocks and Shares ISA on Vanguard, you can only deposit up to £5,000 in Vanguard’s Stocks and Shares ISA account. You can transfer your ISA account across providers, to avoid losing your tax free investments.
  • If you deposit £18,000 on a ISA account and then withdraw all of it, you will only be able to deposit £2,000 on a ISA account after that (on the same tax year4)
  • Every year, the annual ISA allowance will renew, and your investments from the previous year(s) will still have tax free gains. As such, avoid at all possible costs to withdraw your investments for these ISA accounts. Even if your ISA investment was made several years ago, their gains will still be tax free.
  • Any capital gains and dividend gains yielded from your investments made in an ISA account will be tax free.

Cashback

You can earn, while consuming. That is cashback.

The reason why this is possible in the first place, is because products are originally sold at a higher margin price, and cashback mechanisms are essentially a way to clawback part of that extra margin back to you. There are essentially two ways you can claim this: via credit cards (and bank accounts), cashback reward websites.

Credit cards (and bank accounts)

There are several credit cards and bank accounts that allow you to gain cashback. Usually you can expect a 1% cashback rate, save from their respective promotional periods.

  • American Express (Amex) cards are overall the best when it comes to cashback rewards, such as the zero-fee Amex Cashback Everyday Credit Card. Trade-off being that not every establishment accepts Amex cards, so you will need a fall back.
  • Since Amex cards are not accepted on all establishments, it is a good idea to have a backup Mastercard or Visa card, which are widely accepted. Have a browse through these top cashback cards to have an idea of what is available.
  • There are credit cards which have pre-conditions, such as having minimum spend, an associated bank account, maintenance fee, amongst others, so make sure to take these into consideration to calculate your bottom-line
  • There are credit cards which collect Avios instead, such as the British Airways American Express® Credit Card. Unless you plan to use them for travel, use their respective perks (e.g. lounge passes), and have spendings that are leveraged by these points, I would steer away from them. This is because there is an associated monetary value with each of these points, and personally I prefer to have these saving readily available on cash.
  • Remember to pay your credit card spending every month in full, otherwise you will be hit by the gargantuan interest rates (about 30%).

Cashback reward sites

On top of the above credit card cashback rewards, you can clawback another layer of cashbacks via rewards sites. In the United Kingdom, the #1 site is TopCashback. To leverage it, just quickly search on their website if they offer rewards for a purchase you are about to make (e.g. hotel booking via Expedia, which easily gets you 10% cashback), use their link, and once you complete the purchase the reward will be available in a few weeks. This is incredibly powerful.

You can leverage other reward sites, such as the perks provided via Amex, that need to be activated, but personally, I wouldn’t bother too much with them.

Points / loyalty cards

Another level on top of the above cashback methods, is to use points cards on for your favorite merchants / supermarkets, which can either provide you direct discounts, or accrue points that can be exchanged later on by discounts or perks. In general they are quick to set up, and their respective cards can be added to virtual wallets, so there is no need to carry their physical presence. For example:

Watch / Phone Wallet

With all of the above credit and points cards, a wieldy and secure way to have them at hand is to add them to your watch or phone. Google and Apple devices largely support these wallets, and adding them is overall straightforward. One aspect I particularly like about this approach, is that carrying around credit cards using wallet appears feels more secure than carrying a physical card, which if stolen, the mechanisms available to block are mostly leveraged via the card provider, whereas via a virtual wallet, apart from have the card’s access walled by the device’s default access (e.g. pin code), the device can also be remotely wiped, if needed.

Equity investments

Seed investment on companies (such as startups) is not restricted to angel investors and VCs. These investments are now widely available via crowdfunding platforms such as Crowdcube or Seedrs / Republic:

  • These platforms allow you to contribute towards a round of investment in a given company.
  • This is a high risk, high reward / loss business proposition, since there are only very specific conditions in which you can withdraw your investment (e.g. company is acquired, gets IPOed). There is a high likelihood that these companies either take a long time span to exit, or that they will go bankrupt.
  • Many of these are covered by a tax relief, such as Enterprise Investment Scheme (EIS), meaning that you can deduce a percentage of your investment from your taxes. For example, if you invest £100 of a company under the EIS scheme, you will be able to deduce £30 (30% of £100) from your taxes (if you do not expect to pay that amount of taxes, then this would not be useful for you).
  • Commonly there are associated perks provided by the companies, such as redeemable vouchers on their businesses.

Other vehicles

Other investment vehicles you can consider are real estate (e.g. buy to let, REIT), commodity trading (e.g. silver, gold), forex trading (buying one currency and selling another while attempting to profit from the trade), crypto trading, p2p lending. Even though I have some experience with some of them, these are not my main focus of my investment strategy, so I’ll abstain from making many remarks on them.

Final thoughts

The above guide is not extensive, by all means, and there are thousands of ways to put your money to work effectively, but it is important to not go overboard and over optimize, as too much time spent on these will eventually carve away from other meaningful areas of your life, such as overly obsessing over points optimizations. This is an eventual opportunity cost, and you should be aware that these methods are a means, not the end. Optimize just enough, automate as much as possible, reap the rewards, gain options, and enjoy your life to fullest!

  1. Dividend gains allowances, and capital gains allowances.↩

  2. ISA is a tax avoidance mechanism encouraged by the UK government. Tax avoidance is related to avoiding taxes using mechanisms that are legal and are expected to be used. Tax evasion, on the other hand, is the usage of illegal methods to circumvent and evade taxes.↩

  3. Stocks and Shares ISA is very similar to its USA counterpart, the Roth IRA, which has a $6.5k yearly limit, and its withdrawal can only be made at age 59.5 tax free.↩

  4. A UK tax year runs from 6 April to the following 5 April.↩

]]> 2024-08-26T00:00:00+00:00 Personal Local (Llama3 8B) LLM extended with WhatsApp + Obsidian Data https://lopespm.com/machine_learning/2024/06/24/personal-llm.html https://lopespm.com/machine_learning/2024/06/24/personal-llm.html Llama3 8B、WhatsApp Messages、Obsidian Notes、Retrieval Augmented Generation、Text embeddings：

总结：
文章描述了如何建立一个完全离线的Llama3 8B LLM代理，具有访问WhatsApp消息和Obsidian笔记的功能，并能够针对其中任何主题进行对话。通过将所有数据保留在本地，包括WhatsApp和Obsidian的数据，实现了一个个人助手。文章详细介绍了从构建块到实施过程，涵盖了运行离线LLM、增强LLM的知识、文档处理和存储、文本嵌入模型等内容。这个项目展示了潜在的丰富可能性，为个人助手的定制和探索提供了启发。

This blog article will describe the implementation of a fully offline Llama3 8B LLM agent with access to WhatsApp Messages and Obsidian notes (or any other data sources), making it able to hold conversations about any of the topics present there. All data stays local, including the data from WhatsApp and Obsidian - it is in fact a personal assistant.

Source code for this project is available on GitHub.


Motivation & Objective

Why was this project assembled in the first place?

  • I’ve built up a considerable personal database:
    • For more than 15 years I’ve been journaling and writing several notes, keeping them aggregated in the same place as I changed their hosting platform. From simple text files, passing by Evernote, and more recently into Obsidian, into which I migrated all of my written notes 1
    • Most of my relevant correspondence for almost 10 years has been made via WhatsApp, which stores all messages locally
  • I’ve realized how powerful it is to reflect on past events and approach them with the lenses of the new experiences and lessons I’ve been accumulating throughout life, but due to the considerable amount of content present in the above data stores, sometimes it is hard to reach and associate the relevant information I am looking for, even though I try to keep the above relatively organized.
  • And of course, how great would it be to have a personal assistant with access to all of these, in a self-contained offline fashion?
    • Currently, state of the art Large Language Models (LLMs) such as Llama3 are feasible to be run locally on a laptop, which is something that still blows my mind.
    • Pre-trained LLMs can have their capabilities augmented (as we will see below) to serve different custom purposes

All of this combined meant that it would be feasible to host a state of the art LLM in my laptop, with potential access to my personal files, without requiring this sensitive information to be hosted or processed elsewhere, which was something that had quite an appeal for me.

Objective

Build an offline assistant with access to WhatsApp messages and Obsidian notes (or any other data sources), using components that don’t fully abstract the process (similar to the approach I took when creating a deep reinforcement learning agent to play a racing game), to learn more about how LLMs and their infrastructures work.


Building Blocks

Running an offline LLM

One of the crucial pieces is actually being able to run an LLM locally. Below we will explore the hardware requirements, and software we can use.

The more GPU cores and VRAM, the better

When running LLMs on a personal computer, there are two main factors that influence its inference performance and overall capability to even being able to run them: number of GPU cores, and available VRAM (video RAM, this is, memory available to the GPU).

  • The more GPU cores available, the more operations can be processed, and machine learning algorithms are all about just finding patterns in numbers
  • The more (VRAM) memory available, the bigger the model can be stored in memory and be efficiently processed. For example, if we consider Llama3 with 8 billion parameters (more information in this video) 2:
    • One 32 bit float value = 32 bits of memory, this is, 4 bytes (1 byte = 8 bits)
    • 8 billion parameters = 8.000.000.000 * 4 bytes = 32.000.000.000 bytes ≈ 32GB VRAM, for a full precision model (32 bit float)
    • If we would use a quantized model, this is, using lower precision floats to represent the parameters:
    • Using half precision 16 bit float parameters: 16 GB VRAM
    • Using quarter precision, 8 bit float parameters: 8 GB VRAM
    • And if we would use Llama 70 billion parameters, it would amount to 280 GB VRAM (float32), 140 GB VRAM (float16), 70 GB VRAM (float8), 35 GB VRAM(float4)

I personally own a MacBook Pro M3 Max with 40 GPU cores and 128GB of unified RAM (meaning that this memory is available to the CPU and GPU), and it just so happens that Apple Silicon computers (M1, M2, M3 chips) are very capable machines to run inference on LLMs, such as Llama3 8B (full precision), or even 70B models (I’ve personally tested the 70B 4bit model 3 on this machine, and can attest that it runs well and in bounds of memory available, as seen on the calculations above)

Running the inference

There are several methods or libraries that can be used for running a LLM in a personal computer. One of the most approachable ones is ollama, which effortlessly allows you (after a quick installation step), to run Llama 3, Phi 3, Mistral, Gemma offline on your computer.

Another possible way is to load the model’s definition from Hugging Face (think of it roughly as a GitHub for machine learning models), and run them via the transformers library, which vividly encourage you to explore. The amount of freely available pre-trained models to perform tasks on different modalities such as text, vision, and audio is inspiring, to say the least.

For this project, a third option was picked, which was using mlx_lm library, that essentially still leverages the Hugging Face Hub to provide the models, but optimizes execution by leveraging MLX, an array framework crafted specifically for harnessing the power of Apple silicon, which was developed by Apple’s machine learning research team.

Augmenting the LLM’s Knowledge

Now that our LLM is running locally, the question that follows is: how to make it aware of all the contents present in Obsidian notes and WhatsApp messages? We have a few options:

1. Retrain / fine tune a pre-trained LLM

The idea here is to retrain / fine tune the entire model, or parts of it, to make it knowledgeable about our contents.

A pre-trained LLM comes with the advantage of common probabilistic knowledge about a language or set of languages that can be leveraged upon, which was a result of a long and costly training process. As such, a full retraining of the model that would not retain information about its trained parameters would be out of the question. Instead, there are several ways we can leverage it:

  • Freeze the existing LLM layers (all or a selection of them), add also add new trainable layer layers on top of the existing LLM layers, and only train the trainable and unfreezed ones.
  • Fine tune the existing pre-trained LLM, which essentially modifies all parameters of the model with a low training rate, in order to retain as much existing knowledge and capabilities as much as possible, and avoid catastrophic forgetting. Because these models are so massive, they are not particularly sensitive to any given parameters being off by a lot, or a little. Their overall size and complexity make up for it, so we can mostly get away with rough updates. To fine tune a model, there are essentially two ways to go about it:
    • Load all of the model’s parameters into memory, and modify them directly during the learning process
    • Take advantage of the fact that you can multiply two small matrices to get a larger matrix (which is the main bread and butter of deep learning models), so we can try to find out the two smaller matrices instead, meaning that we only need to maintain those in memory. All the parameters of the model are still modified during training, but we don’t need to hold them all in memory at once. That’s what Low-Rank Adaptation (LoRA) fine-tuning method does, and it is one of the most popular ways to fine tune LLMs, along with its QLoRA counterpart (which uses a data type that reduces by 4x the already lower LoRA memory footprint).

Some important aspects of these techniques:

  • We are compressing our documents into the LLMs internal knowledge, and we have less control of how precisely these models retain the most accurate information. These could lead to hallucinations.
  • If the data we want to augment our LLM with is dynamic in nature (which for our case, I would prefer it to be, so that the assistant can have access to the most relevant information when used), this would mean that the model would need to be fine tuned every time an update should be made.
  • The training data needs to be structured in a way that makes sense for the model to be used. For example, WhatsApp messages and Obsidian notes would need to be transformed into a sensible training set that would fit the tone I would aim to have for the assistant.

Given that I would want to experiment with different types of assistants and different ways to leverage and explore the data in a conversational way, and have access to the most up to date information available, which could be available in several different formats (for example, let’s say I would like to augment the LLMs knowledge with historical CO2 readings of my apartment), retraining / fine tuning the model was not a very inviting approach.

2. Insert all documents into the prompt, as context

The idea here is to provide all the context that LLM needs, in the prompt’s context. For example:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a AI assistant which answers the user's question based on your prior knowledge
and a collection of the user's documents.

Each of the documents has the format <<document:||document_name||>>||content||.
||document_name|| represents the document name. ||content|| represents the
document's content. Don't show the <<document:||document_name||>> portion to the user.

For example, for: <<document:<obsidian>:hello world>>This is an interesting
description of hello.
- ||document_name|| is <obsidian>:hello world
- ||content|| This is an interesting description of hello.

These are the user's documents:
----------
<<document:<obsidian>:lamas running down the street>>The content for lamas
going down the street
<<document:<obsidian>:alpacas running down the street>>That time when I saw a
alpaca running down the street.
----------

<|start_header_id|>user<|end_header_id|>Tell me about that time I've seen animals
running down the street<|eot_id|>


This would be the most convenient approach, but the context window available to a LLM, the number of tokens it can use at one time to generate a response, is limited.

Llama3 8B for example, allows for a 8K token context window, and there have been attempts to extend it further to 80K or 262K, and Google’s Gemini 1.5 Pro can handle a 1 million token context window, for example.

Given that my Obsidian notes total around 8M characters in content, and my WhatsApp messages around 16M, even if we assume that an average of 1 token per 4 characters, just the Obsidian notes would clock around 2M tokens, which makes this method infeasible

3. RAG (Retrieval Augmented Generation)

To navigate around the context window limitation, instead of inserting all the documents into the prompt’s context in one go, we can instead be selective on the information we want to provide on the prompt, depending on the user’s request(s).

For example, if the user asks the AI assistant “what do you think about that gaming project I did 4 years ago?”, behind the scenes we can query a database, and ask it the most relevant results about gaming projects that happened about 4 years ago, and then use those documents content into the prompt’s context.

That’s the overall idea of RAG.

Now, many questions come up:

  1. Transforming a conversation into a query: How do we transform the user’s prompt + the context of the conversation, into a query that asks our database exactly what the main LLM needs to provide a contextual reply?
  2. Fetching contents related to a query: How can the content database know which are the most relevant content related to a given query?

Let’s explore these in the next section.

What makes RAG tick?

1. Transforming a conversation into a query

In order to produce a query that transforms an entire conversation into a concise sentence that can be used to get the documents in the database that are most similar to it, we can feed in the conversation into a LLM specialized in creating summarized queries. Here is an example of prompt we can use for it:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>   
You are the user's document keeper that has access to all the documents for the user.
Based on the entire conversation only, you will only reply with the question you
will ask to query the appropriate user documents, in order to get all the relevant
user's documents related to the conversation.
<|eot_id|>

<< entire conversation >>

<|start_header_id|>document keeper<|end_header_id|>


This is how it would it would look like with actual information about the conversation:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>   
You are the user's document keeper that has access to all the documents for the user.
Based on the entire conversation only, you will only reply with the question you
will ask to query the appropriate user documents, in order to get all the relevant
user's documents related to the conversation.
<|eot_id|>

<|start_header_id|>assistant<|end_header_id|>
Hi there! I\'m a helpful AI assistant with access to your documents. What can I do for you today?
<|eot_id|>
<|start_header_id|>user<|end_header_id|>
What is the name of the first steam game I developed?
<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>
Survival Ball, which you published in 2018
<|eot_id|>
<|start_header_id|>user<|end_header_id|>
which interesting things could you tell me about it?
<|eot_id|>

<|start_header_id|>document keeper<|end_header_id|>


This LLM’s reply would be Which are interesting facts about Survival Ball, a game developed in 2008?.

Now that we have a query, such as “Which are interesting facts about Survival Ball, a game developed in 2008?”, how can we get the documents (Obsidian notes and/or WhatsApp messages) that are most relevant with that query?

One possible way would be to perform a custom decomposition of the query phrase, and attempt to look for documents that most referenced similar words to the query. This custom search would be brittle, require careful heuristics to match these, and would not completely take into account what that sentence means, and therefore the matching between query and documents would be likely unreliable.

Instead, we have a much more powerful concept at our disposal, which will help us perform this matching: text embeddings.

Text embeddings

The idea behind text embeddings is that if we transform a given piece of text into a vector that is in close proximity to other vectors that have a similar meaning, then from a given query text we can get the most conceptually similar pieces of text to it. 4

For example, if the vectors representing our text were trained to be closer together when they rhyme, then close to the word “dog”, we might have the word “fog” closer to it, but “cat” further away; and conversely, if we trained these to represent their conceptual meaning, then for the word “dog”, we might see the word “cat” nearer to it, but “floor” further away 5

A mind-boggling implication is that we can perform algebraic expressions on these vectors. For example, if our vectors were trained to represent the meaning of the text we could have the following:

  • The classical example is that if grab the vectors for king, woman and man, and perform this operation: (king + woman) - man, the closest vector could be queen
  • (London + Japan) - England -> Tokyo
  • (boat - fly) - airplane -> sail
  • (cow - oink) - pig -> moos

Likewise, if we probe which are the closest pieces of text related to dog, we could get dogs, puppy, pit_bull, pooch

These examples were takes from an actual working word2vec model that was trained on a google news dataset, which I invite you to explore in this Google Colab notebook.

Exploring the above examples using a word2vec model in Google Colab

For our project’s objective, the most relevant property is the latter one. Given query text, we would transform it into a vector, and get the closest vectors in our document (vector) database, which in our project will be supported by a PostgreSQL database using the pgvector extension. More about this below.

Putting it all together

With the prior knowledge for the previous section, we are now in a position to build an system architecture that would best fit our objective:

Personal LLM Architecture

The overall idea is that we have a Llama3 8B LLM powering the main AI assistant, which has its knowledge augmented via RAG (Retrieval Augmented Generation) by using another Llama3 8B LLM to generate queries used to retrieve documents from a vector database powered by PostgreSQL. These retrieved documents are then added to the main LLM’s context, and the LLM generates a response based on the retrieved documents + user’s prompt + ongoing conversation. Below are further details for these.

  1. First prepare documents and store them in a vector database.
  2. and then, when using the chat interface, evaluate the user’s input and conversation, query the vector database for relevant documents, add them as context into the main LLMs prompt, which will output the response that the user will see.

Document preparation and storage

Before anything else, all Obsidian notes and WhatsApp messages are exported, parsed, transformed into text embeddings, and inserted into the vector database.

1. Exporting the data sources

No export procedure is required for Obsidian, since it stores its text contents inside markdown (.md) files, we just need to point directly to the folder we want to target, which can be a vault’s root folder, for example.

For WhatsApp messages, a result.json holding a structured JSON dump of all messages and their respective metadata, is generated via an open source exporting utility which extracts all messages from the local WhatsApp database present in the mobile device; and a contacts.csv is exported from all Google contacts, in order to enrich the context of the aforementioned messages. A step-by-step process to generate these is described in the repo’s Readme

2. Parse, vectorize and persist

With our data sources ready, we can now start extracting relevant information, transforming the relevant content into a vector, and then persist into a vector database. The entire process is made by running:

$ python create_and_persist_embeddings.py


Each of the separate Obsidian note paragraphs and WhatsApp messages are accumulated into a single entry, so that locally close contextual content can be readily available and provided to the main LLM AI assistant, when retrieved, so that it has better immediately relevant contextual information about a given topic. It was chosen to accumulate a set of note paragraphs / messages that in total would not surpass the maximum content length of the model used to vectorize the content, which in this case has a maximum content length of 514 (see how to get the maximum content length here).

All of these are batch processed, so that vector embedding creation can be done much more efficiently. Given that in total there were 25M characters worth of content across these two data sources, these optimizations make a substantial change in the processing run time. Further optimizations could be made in optimizing DB insertion commits and streaming each of the data sources contents, instead of storing them entirely into memory.

Obsidian

The python script calls into a custom module that traverses through all the markdown (.md) files inside Obsidian’s root folder and its respective descendent folders, and accumulate a group of lines in a given file into a distinct entry.

For example, a Obsidian note called Survival Ball TV.md with this content

Only had to take OUYA project, change some resources to match Android TV controllers, detect Nexus Remote.
Test on the Nexus Player
Delete all OUYA related resources
Most of the time was spent (...)


Would be broken down into to two separate database entries 6:

  • Entry 1.
    • DB ID of the entry
    • Content: Only had to take OUYA project, change some resource (...) - these would be first lines of the note
    • Source: <obsidian>:Survival Ball TV
    • Embedding: the vector representation the content text (content attribute above)
  • Entry 2.
    • DB ID of the entry
    • Content: Unity is pretty great, and has tons of really great stuff (...) - the lines coming after the ones in Entry 1. above
    • Source: <obsidian>:Survival Ball TV
    • Embedding: the vector representation of the content text (content attribute above)
Sample Obsidian entries in the vector database 6

WhatsApp

Likewise, the script also calls into a custom module that traverses all the messages, and similarly to the approach taken above for Obsidian notes, a set of messages are aggregated into a single entry.

For example, a WhatsApp conversation on the “Deeper Conversations” chat group would be broken down into several different entries, such as [^6]:

  • Entry 1.
    • DB ID of the entry
    • Content: message from "Deeper Conversations 💭" to me, on 2021-06-26 10:15:44: And that’s absolutely normal (...) - several lines of the conversation would be aggregated here
    • Source: <whatsapp>:xxxxxxxx-yyyyyyyy@t.as (the numbers were redacted)
    • Embedding: the vector representation of the content text (content attribute above)
  • Entry 2.
    • DB ID of the entry
    • Content: message from "Deeper Conversations 💭" to me, on 2021-07-04 18:37:54: Challenge yourself (...) - another group of several lines of the conversation would be aggregated here
    • Source: <whatsapp>:xxxxxxxx-yyyyyyyy@t.as (the numbers were redacted)
    • Embedding: the vector representation of the content text (content attribute above)
  • Entry 3., 4., etc would have a similar structure to the ones above
Sample WhatsApp entries in the vector database 6

3. Storage (Vector Database)

The storage of the above entries is handled by a PostgreSQL database using the pgvector extension, which allows PostgreSQL to store and query vector attributes, and notably for our project, it provides tha capability to query for the closest vectors / entries related to a given query vector via cosine similarity 7.

The database is hosted via a docker container, which makes it easily encapsulated and quick to set up and run.

Embedding model

As we discussed above, embedding models can be trained have their embedding representation to be optimized and better suit the problem at hand. In general, there are several types of tasks a embedding can be better or worse at, so testing each single model against our problem would certainly inefficient.

Fortunately, a Massive Text Embedding Benchmark was been made and its results are freely available in this dashboard, which compares several models against capabilities such as classification, clustering and reranking.

Massive Text Embedding Benchmark (MTEB) Leaderboard

The model used for this project is intfloat/multilingual-e5-large8, which was chosen for its:

  • Reasonable embedding dimensions (1024), which essentially means of many dimensions will the embedding vector have, allowing for a reduced database footprint and faster retrieval when compared to higher dimension models (such as GritLM-8x7B with 4096 dimensions)
  • Good average score across the all tasks (ranking 20th overall)
  • And most notably support 100 different languages, one of them being Portuguese, which is quite relevant since my notes and messages are largely a mix of English and Portuguese, with some sprinkles of other languages (which this model also supports)

Related (source code here)

Evaluation phase

Having all the documents vectorized and available in a database, we can now jump into the evaluation phase, in which we leverage upon these on chat conversation with an AI assistant.

To start the chat session, we only need to activate the respective conda environment 9 and execute:

$ python main.py


1. The main AI Assistant LLM

Upon starting the above script, a Llama3 8B model (mlx-community/Meta-Llama-3-8B-Instruct-4bit) is loaded into memory and is instructed to behave as AI assistant / philosopher / psychotherapist (or any other role one so wishes, as seen below, and the user is greeted with an initial message, and is able to start inserting a prompt.

Greeting message from the AI assistant

This will be LLM having the conversation with the user.

2. Transform conversation into DB query

Once the user inserts their prompt, the entire conversation is provided to an auxiliary “document keeper” LLM, which has only one job: transform the entire conversation the user is having with the main AI assistant into concise sentence that aims to represent the information we are interested to provide as context on the upcoming reply.

The document keeper LLM produces a query sentence using the previously described process (related source code), and then transforms it into a vector using the embedding model.

Now we have everything we need. Using this vector representation of the query, we can ask the database which are the top 10 entries which have their respective vectors closest to the query vector.

So for example, if the text query created by the “document keeper” LLM is Which are interesting facts about Survival Ball, a game developed in 2008? (resulting in a vector representation of [1.3, 4.2, 3.3, etc]), then a SQL query such as this one would be made against the DB:

SELECT id, content, source, 1 - (embedding <=> [1.3, 4.2, 3.3, etc]) AS cosine_similarity
FROM items
ORDER BY cosine_similarity DESC LIMIT 10,


3. Retrieved entries are added as context on the main AI LLM prompt

Now that we have the retrieved a set of relevant entries we want the main AI assistant LLM to consider towards its reply, we now can add them as context for the prompt for the main AI assistant LLM, very similarly to what was described above:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a AI assistant which answers the user's question based on your prior knowledge
and a collection of the user's documents.

Each of the documents has the format <<document:||document_name||>>||content||.
||document_name|| represents the document name. ||content|| represents the
document's content. Don't show the <<document:||document_name||>> portion to the user.

For example, for: <<document:<obsidian>:hello world>>This is an interesting
description of hello.
- ||document_name|| is <obsidian>:hello world
- ||content|| This is an interesting description of hello.

These are the user's documents:
----------
<<document:<obsidian>:lamas running down the street>>The content for lamas
going down the street
<<document:<obsidian>:alpacas running down the street>>That time when they saw a
alpaca running down the street.
<<document:<whatsapp>:aaaaaaa-bbbbbbbb@t.we>>message from "Thy Table Turner" to me, on 2024-03-26 13:15:44: I've just seen a alpaca shuffling down the street
<<document:<whatsapp>:aaaaaaa-bbbbbbbb@t.we>>message from me to "Thy Table Turner", on 2024-03-26 13:16:32: Pretty sure it was lama though
----------

<|start_header_id|>user<|end_header_id|>Tell me about that time I've seen animals
running down the street<|eot_id|>

4. Output from the AI assistant LLM presented to the user

We now have everything we need, and the prompt is ready to be evaluated by the main LLM.

Once the prompt is evaluated by this LLM, we present the output to the user, and then we cycle through the entire evaluation phase again once the user submits their request.

Results

Example interaction with the AI LLM, having its role as “AI assistant”

Overall, I’ve found interactions with this AI to be quite interesting, even though sometimes it can output blobs of information that could be cleaner or have better answers. The former can be addressed by tweaking the prompts (i.e. prompt engineering) having increased efficacy towards the behavior I aim the Ai to have, and the latter by either cleaning Obsidian notes directly, or improving the data processing and extraction. Regardless, I find these results to be quite pleasing, useful and even amusing.

For example, the code allows you to choose the AI’s persona by changing this constant

The AI’s persona is customizable

When choosing the AI’s persona to be of a AI “psychotherapist”, this happens (disclaimer: please consult a professional if you require therapy, this agent is meant for pedagogical / entertainment / utilitarian uses only):

Example interaction with the AI LLM, having its role as “AI assistant”

Final Notes

Adding other data sources

This project focuses on using Obsidian notes and WhatsApp messages to augment the main LLM, but virtually any piece of content could be used to augment it. CO2 historical readings, tax documents, financial statements, diplomas, official documents, calendar events, etc. Sky is the limit.

Doing so in practice would be as straightforward as adding an additional module to this script, and have it return the pieces relevant pieces of content, and then re-run:

$ python create_and_persist_embeddings.py


Inspiring wealth of possibilities

This type of assistant has been something I’ve been daydreaming for a long time, and having the peace of mind that I have full control and customization of it is something that profoundly satisfies me. Not only that, this entire project has an immense learning opportunity for me, and as I researched through it and pieced the puzzle pieces together, I’ve felt humbled and inspired by the explosive amount of possibilities that these models provide.

For example, just browsing through the wealth of freely available models, ready to be used, combined and assembled into different scenarios, leaves me at awe:

A sample of different classes of models available in Hugging Face. Each of those having dozens of models, sometimes thousands. All freely available.

Further references

I find it fulfilling to learn about a subject by fundamentally understanding its core components, hence the usage of several lower level components, similar to how I previously built a deep reinforcement learning agent), but there are several readily available tools and libraries that handle a lot of the heavy lifting:

  • LangChain: framework designed to simplify the creation of applications using large language models (LLMs). As a language model integration framework, LangChain’s use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis. This could have been used to abstract most, if not all, of the RAG heavy lifting, which is quite useful.
  • Other Vector Databases: although PostgreSQL worked great for this project, there are other possible, such as Pinecone (this is a externally hosted DB, hence why it was not considered), Qdrant, chroma, faiss or MongoDB
  • Replicate: Similarly to Hugging Face, Replicate is a platform that collects various open-source large language models (LLMs), offering AI models for a variety of purposes, including AI text generation, AI image generation, and AI video generation. Developers can use these models through Replicate’s API.

Source code

All source code is available on GitHub at https://github.com/lopespm/personal_llm


  1. Obsidian is an amazing platform / app which I vividly recommend. I’ve recently migrated all of my notes from Evernote into Obsidian, and couldn’t be happier. Obsidian stores all of the notes in plain .md files that are yours to keep, and allows you to completely customize it to your heart’s desire. For example, I created a custom script that packages all of the notes, encrypts it, and stores that vault into my Google Drive. I would be happy to provide more details about this in a posterior note.↩

  2. Note that these calculations are based on loading + inference for a batch size of 1. Training a model required far more VRAM↩

  3. Specifically, I’ve used the model available on hugging face at mlx-community/Meta-Llama-3-70B-Instruct-4bit↩

  4. The process of how these embedding models are trained is a fascinating topic by itself. The idea behind it is that you place a given piece of text on the model’s input and on its corresponding contextual word on the output, and then train the model to attempt to correctly predict the output text. For example, if using a word2vec skip-gram architecture, the input is a single word and the output is trained to be the words around it. The model’s number of weights is lower than the number of pieces of text in the dictionary, so that the model is forced to compress these associations into its weights. Once the model is trained, we use its learned weights to generate an embedding for a given piece of text. When a model is trained on large swathes of text, this simple method yields incredible results.↩

  5. In practice, representing these vectors as being closer together when they rhyme, vs when they are conceptually closer, can be achieved by training the embedding model on poetry books and giving higher emphasis on the last words of each line for the former, and training the embedding model on more generic text datasets, such as wikipedia, and using samples around the seed word uniformly as target words (using a skip-gram architecture for example) for the latter.↩

  6. These screenshots were taken from DBeaver, and open source universal database tool which is incredibly powerful and well designed.↩

  7. The pgvector extension supports L2 distance, inner product, cosine distance, L1 distance, Hamming distance, and Jaccard distance.↩

  8. Related backing “Multilingual E5 Text Embeddings: A Technical Report” paper for this model is available in arxiv↩

  9. Conda provides for a self-contained and easy way to manage packages and environments. Whereas pip is a package manager and virtualenv is an environment manager, conda is both. Here is an illustrative comparison table↩

]]> 2024-06-24T00:00:00+00:00 Aloof https://lopespm.com/notes/2024/06/09/aloof.html https://lopespm.com/notes/2024/06/09/aloof.html Einstein, ramble, safety, environment, agency 总结:

文章讲述了爱因斯坦的故事和作者自己的体会。爱因斯坦常常徜徉在自己的思绪中，而作者则喜欢在漫步中思考问题、策略和得出结论。他认为这种思维过程的基础是安全感。环境对一个人的影响很大，不安全的环境会限制思考和行动，而拥有选择的权利则能让人有更多自主权决定生活的方向。因此，我们应该尽量控制可以控制的部分，做出对长远有利的选择。

Einstein’s rambles

I’ve been listening to the “Einstein: His Life and Universe” audiobook, from Walter Isaacson, which recounts a series of curious stories of how he carried himself through his life.

Einstein was often known to be aloof and lost in this thoughts, so much so that when sailing his clunky 15-foot sailboat named Tinef (Yiddish for “worthless” or “junk”), he often found himself to being rescued more than once by locals near to his rented a summer cottage overlooking Cutchogue Harbor.

Another delightful story was when he lived in Princeton, where occasionally, he would take rambling walks on his own. One day, called into the Institute for Advanced Study, where he worked, and asked to speak to a particular dean. When the secretary said that the dean wasn’t available, he hesitantly asked for Einstein’s home address. That was not possible to give out, he was informed. His voice then dropped to a whisper. “Please don’t tell anybody,” he said, “but I am Dr. Einstein, I’m on my way home, and I’ve forgotten where my house is.”

Thinking, in safety

Over the past years, I’ve made a habit of having at least one long walk / hike every weekend. I take immense joy - if I happen to go by myself and I’m not actively listening to an audio book or am concentrating on some task - to just wander around and immerse myself in random thoughts, think about hard problems, strategise, and attempt to arrive at sound conclusions. This mental wandering, almost like a meditation, I am sure that happens for most of us, in several instances. Maybe when waking up, during the shower, during a walk, during a boring talk, when contemplating art or nature, etc.

Since we are not omniscient and omnipresent, we have several limitations in knowing how a given action will translate into a set of consequences, so we use probabilistic models and heuristics in our thought processes to discern which are the most likely actions that will help us arrive at a desirable outcome. And the more relevant information and time we have to process our thoughts, the better our results will be.

In all of these, I believe there is one common denominator that is fundamental for that process to happen: safety.

The influence of one’s environment

Let’s assume you live with access to information and resources (food, money, shelter, etc), but are living in a situation where your surrounding environment requires you to keep alert and attentive to hazard to your physical / mental wellbeing, or you are in a situation where your social stability is dependent on incapacitating amounts of your energy and time. This could be due to living in a dangerous neighborhood, having overly demanding peers, an abusive household, etc.

Even though there might be a series of limited actions one could take to improve their situation, the constant fight-or-flight severely restricts the opportunities where these action plans can be considered, which eventually leads to reactive actions that could have a number of possible outcomes (good or bad), rather than surgical long term actions that would pave the path towards positive outcomes.

The takeaway

While many many situations are not under our control, such as our initial conditions, birth place, etc, which place a default upper bound on any possible outcomes, we do have agency over some parts of our life, and that is what we can focus on. The path to unwanted consequences is a slippery slope made of several small decisions, some under our control, others not. But for the ones we have the opportunity to control, we should.

For example, the peers one choses to have around, the choice to avoid short term pleasures that have a high long term cost, the choice of which country or neighborhood to live in, the choice of which job offer to strive for, etc.

Do notice that the italics are intentional. The choices above assume that those are options you have under your control to pick, which I am conscious that is not always the case.

]]> 2024-06-09T00:00:00+00:00 Let me tell you a story https://lopespm.com/notes/2024/05/05/let-me-tell-you-a-story.html https://lopespm.com/notes/2024/05/05/let-me-tell-you-a-story.html 故事 概念 信念 社会 领导者

总结: 人类是唯一能够使用语言来创造虚构概念的物种。通过故事，人们能够传达和认同共同的概念与信念，进而形成社会共识。故事是传播思想、理念和规则的有效工具，可以帮助领导者激励团队、影响社会，并塑造未来。有效的讲故事能力对于解决重大问题和推动变革至关重要。

You could never convince a monkey to give you a banana by promising him limitless bananas after death in monkey heaven. How do you cause people to believe in an imagined order such as Christianity, democracy or capitalism? First, you never admit that the order is imagined.

― Yuval Noah Harari

Take a minute to think of what a nation is. Let’s say the United States of America. What makes someone American? Is it their physique, genetic traits, family lineage, pheromones? No, it’s a concept in their head, which stems from a mix of identity, ideology, personal gains from abiding to that concept (say, gaining currency via a job or a company, that allows you to acquire things you desire), perceived protection and support from peers because they believe you are commonly aligned on core beliefs / concepts.

The same can be said of a company. A company is not a physical being, it’s a shared idea in our imagination. As Yuval mentions in Sapiens, Peugeot does not really exist, except in our minds. It doesn’t have a body, mind, anything. We have the common belief that Peugeot exists, because we also have the common belief of what a company means, we are aligned on how it should be represented and who can represent it, we recognize it as an entity with rights and duties, and which laws it should abide by.

Laws, as well, are worth nothing if only one person believes in them. They are powerful because a large group of people believe in them, and they expect others to do so as well. That’s why we have courts, prisons, and educational systems for that matter. They are ways to punish, format, and incentivise for the entire group to have the required level of consistency, and keep rowing in the same direction.

That’s why complete anarchy loses its edge on a large scale. It extinguishes the biggest thing Humans have going for them: the ability for large numbers of people to rally in common goals, even though they don’t know personally most of their fellow believers.

As N. K. Humphrey suggests, these ideas, behaviors, or styles that spread by means of imitation from person to person within a culture, also known as memes, could even be considered as living structures, not just metaphorically. The generational propagation of a meme outweighs the lifespan of the Human who created it, so in a way, “we are living inside the dreams of dead people”, as Yuval puts it.

That being said, how do we align on a common concept or belief?

Stories

We’ve seen from above how powerful and valuable having a broad range of concepts aligned within a group of people. They bring value to the whole, but also to the individual.

You can force someone to abide to a given concept via force (coercion, blackmailing, etc), but since they see little value going on for them, they don’t have the incentive to grow that concept inside them, or to propagate that concept to others

Stories however, allow you to describe a world to others that they can interpret in their head, and they can see by themselves how it would benefit them, and the ones around them. It’s an information passage mechanism that allows Humans to communicate and agree on a given concept, and they are everywhere:

  • Children’s bedtime stories - notice how several of these transmit moral values, this is, expectations of how one should behave in a given situation to be in alignment with the overall grpup / society
  • Novels, biographies, movies
  • When interviewees tell you a story of their childhood, how they created their company, how they achieved X, what happened during their struggles, etc
  • When a leader, such as a CEO, convinces others of their vision / story, and that future prospects that would derive from working in a certain direction
  • When meeting someone, when you talk about your background and your goals
  • When you sell you employee skills to an employers, giving a summary of your professional trajectory
  • When politicians portray their vision on how they want to improve their nation
  • The lore of a nation, tribe, family, and their laws / duties / rights

Being a good storyteller is a powerful skill

As a species, one of our biggest distinguishing factors is that we are able to goal on highly abstract concepts that only exist in our minds, such as the concept of a company, a nation, a mission, an ideology, an intellectual relationship, the concept of money. As Yuval puts it, “We are the only species with the ability to use language—not just to describe things we can see, taste, and touch, but also to invent stories about things that don’t exist.”

A magical thing happens when a group of people interpret and value that concept in the same way, and are highly aligned on it. Narrative is the first and most crucial step for leaders who want to solve big problems and shape the world for the better. After all, a person can have the greatest idea in the world, but if they fail to rally others to make that idea come alive, nothing gets done.

]]> 2024-05-05T00:00:00+01:00 Banishing mobile phones from bedroom https://lopespm.com/notes/2024/04/01/banishing_mobile_phones_from_bedroom.html https://lopespm.com/notes/2024/04/01/banishing_mobile_phones_from_bedroom.html Braun alarm clock, Onyx BOOX Note Air3 B/W, distractions, focus, productivity 总结:

文章讨论了作者如何改变自己的习惯来提高专注力和生产力。作者根据克里斯·威廉姆森的建议，改变了不在卧室使用手机，并使用Braun闹钟和Onyx BOOX Note Air3 B/W来替代手机的闹钟和笔记功能。通过这一改变，作者发现早起后拥有更多的时间来计划、反思、写作、阅读和放松，最终提高了日常的专注力和生产力。这一变化对作者产生了积极的影响，值得其他人尝试。

The Problem / Why

When asked about which practices supported Chris Williamson into his current position, Chris mentioned that sleeping with his phone outside of the bedroom and getting a plain alarm clock was his number one change.

I found that statement interesting, especially because I had grown a habit of seeing the occasional video before sleep and being engaged with random distractions shortly after waking up.

Overriding these behaviors required a good amount of self discipline and willpower to be overridden, and fundamentally deviated my focus from what I would want to accomplish on a given day, from the book I had been putting off, or to simply wind down.

Alternative Alarm and Note Taking

Even though my phone sponsored my distractions, it was also my trusty alarm and favorite way to write down notes and reflections that crossed my mind outside my work desk. So to make the leap of keeping my mobile phone at the bedroom door, I would need to replace the alarm clock, and find a new way to write down my notes.

Alarm

This seems like an easy step, but it turns out the selection of alarms is immense, and many of them either do too much, or don’t tick all the boxes.

Essentially, I needed a trustworthy, low maintenance, alarm with essential functionalities that would not flood my dark bedroom with blue light when I wanted to check the time during night.

I essentially browsed through:

  • Lumie alarms: had the enticing proposition of science-backed approach of waking up via warm light or soothing sounds, but didn’t have the basic functionality of having recurrent alarms on the models I was interested in, and required a mains connection, meaning that upon a power outage it could only support itself for a limited amount of time.
  • White brand alarm clocks: the selection is immense on Amazon, and even though they come in at relatively cheap prices, some require a mains connection, or have short battery life, or need to be recharged every so often, or do too much (like having a bluetooth connection) or use noisy clock dials.

After several searches, I’ve stumbled upon Braun alarm clocks. These have been around for decades, and I recalled that Steve Jobs was inspired more than once by products made by Braun when creating his Apple products, due to Braun’s thoughtful approach on how everyday products were designed.

I’ve settled with the Braun BC08B. No moving parts, a single AAA battery apparently lasts more than half a year, small, simple, easy to operate, and the beeping sound is not brain drilling. The one aspect I am not fully satisfied is its screen light, which when activated is too strong on a dark bedroom (my mobile phone far excels in this aspect, due to its less intrusive always on black and white display).


Note Taking

These were the alternatives I found for writing down notes, and jotting down diagrams (as a plus):

  • Physical Notebook:
    • Pros: Easy to use, low cost, no distractions, low maintenance (no battery to recharge, software updates, etc)
    • Cons: My handwriting is appalling and my character output is much lower when compared to typing; extra step needed to transfer it a digital means; harder to organize
  • E-Ink Devices:
    • Remarkable 2:
      • Pros: well thought out, clean and focus driven, great writing experience; allows for handwriting to text conversion
      • Cons: Able to connect to a Google Drive for free, but works storage rather than syncing, so it is cumbersome to iterate on notes and diagrams between it and other devices
    • Supernote:
      • Pros: clean and focus driven, great writing experience; allows for handwriting to text conversion; open roadmap and ideology; easy repair and upgrade
      • Cons: As with Remarkable, it does not provide real file syncing, but file copy to Google Drive
    • Kindle Scribe:
      • Pros: Writing experience seems to be good and on-device handwriting recognition to convert notes into actual text
      • Cons: No Google Play apps, meaning that Google Keep and Google Docs would not be available, and no Google Drive sync; the device caters more to readers, rather than writers in my opinion
    • Onyx BOOX Note Air3 B/W:
      • Pros: it can use Google Keep and Google Docs apps, since can do almost everything a normal Android table can, but since it uses a e-ink screen and has limited processing capabilities, it does not make a good fit for browsing the web or seeing videos (which is actually a good thing); the default experience and drawing app are bloated offer too many distractions (see here how to debloat it); allows for handwriting to text conversion within Onyx drawing app
      • Cons: noticeable lag when writing; if not using Onyx’s cloud, it requires extra steps to export and share a diagram / drawing
  • iPad Pro: even though this device is amazing, it made it too easy to access the same distractions I was trying to avoid, and it also sports an invasive backlit screen that provides too much stimulus while in bed. A hard no.

Given the options above, I picked the Onyx BOOX Note Air3 B/W. Note that the color screen version of this tablet is available, but I did not pick it because of Air C’s reduced battery life from the always-on BSR (Boox Super Refresh Technology). BSR allows for a smoother and more responsive experience when writing for example, but that is not actually a problem since I mainly use the Boox to write long streams of text / braindumps, rather than text editing; and BSR also takes it a step closer to a general purpose tablet non-eink display, which starts to defeat the initial purpose. Apart from that, I have a personal preference towards the black and white screen feel.

Learn more on how I customized my Boox Air3 here.


The Result

I have now transitioned to not having any mobile phones in my bedroom, and only having the Boox Air 3, the Braun alarm clock, physical books and my Kindle. I am pretty satisfied with this change, which I’ve noticed to provide more focus time after waking up, more time to think, plan my day, reflect, write my thoughts, read, and wind down.

The most fundamental aspect was having more time to focus on my plan for the day, which translated into a substantial improvement on how productive and consequential my days have become. I welcome anyone to give it a try!

]]> 2024-04-01T00:00:00+01:00 Minimalist Onyx BOOX Note Air3 B/W Setup https://lopespm.com/notes/2024/03/25/minimalist_boox.html https://lopespm.com/notes/2024/03/25/minimalist_boox.html As part of the plan to banish mobile phones with access to instant rewards from my bedroom, I’ve acquired a Onyx BOOX Note Air3 B/W, which is a Android tablet with a black and white e-ink screen that is focused on writing and reading 1

House Rules

First of all, why I did get the device in the first place? This was in order to:

  • Reduce distractions before going to sleep and after waking up, increase focus on the things that really mattered, block out all other noise, and make it really hard to have access to instant gratification and rewards (such as video consumption)
  • Register any ideas and thoughts that occur to me before and after going to sleep, which is one of my most proficuous times for such, and then have them available on other devices

How to achieve this using the BOOX:

  • Only two functionalities should be easy and quick to use:
    • Write text and notes (via Google Keep and Google Docs)
    • Write diagrams and mental maps (via the device’s main app, which has the snappiest performance)
  • Getting to above functionalities should be as fast as possible, and they should be fast
  • The cost to get to other apps and functionalities should be high

Focusing & Debloating

The Onyx BOOX Note Air3 B/W is not an incredibly fast device, and comes with several functionalities, apps, and a launcher that increase the distraction degree, and also slow down the device.

Google Play, Google Keep & Docs

The first step towards the objective above was to install Google Play (see guide here, and afterwards installed these apps:

  • Google Keep
  • Google Docs
  • Android Hidden Settings - this app provides access to several hidden settings, that allow the access of settings such as enabling the Developer Options in Android (useful in the debloating process described below)
  • Lawnchair (see below)

New Launcher: Lawnchair

Before: Default BOOX Launcher 2

After: Using Lawnchair Launcher

The default device’s launcher was replaced by Lawnchair, which is a lightweight, open-source, highly customizable launcher for Android.

What was added to the Lawnchair’s home screen:

  • Shortcut to Google Keep
  • Shortcut to Google Docs
  • Shortcut to Onyx Launcher - for any custom changes that are only accessible on the device’s default launcher, which are are in practice, so this was eventually something I removed from the home screen as I wrote this note
  • Shortcut to Google Calendar - to add any events or reminders
  • A widget to Onyx Sketch app, which is one the only way I found to have that app directly accessible from the Lawnchair home screen, without having to use Onyx’s launcher to do so. The widget can be added by long pressing on the home screen, then select “Widgets”, and then on the “ContentBrowser” section select one of the “Notes” widgets

Disabling bloatware

With all of the above in place, we could start disabling non-essential apps and services, to prevent them from running in the background.

Here are the steps:

  1. The first step is to enable “Developer Options”, by accessing the system settings via the “Android Hidden Settings” app above, and then finding the Build Number in order to click it enough times to eventually enable “Developer Options”.
  2. Connect the BOOX to the computer
  3. On the computer’s terminal run the below commands

In general, these are the commands one can run via the terminal to manage bloat apps:

// See all packages
$ adb shell pm list packages

// See all disabled packages
$ adb shell pm list packages -d

// Disable a package
$ adb shell pm disable-user --user 0 <package_to_disable>

// Re-enable a package
$ adb shell pm enable <package_to_disable>


And these are the specific ones I used in my BOOX to disable bloatware, without any relevant side effects for my usages:

adb shell pm disable-user --user 0 com.onyx.appmarket
adb shell pm disable-user --user 0 com.onyx.floatingbutton
adb shell pm disable-user --user 0 com.onyx.dict
adb shell pm disable-user --user 0 com.onyx.mail
adb shell pm disable-user --user 0 com.onyx.igetshop
adb shell pm disable-user --user 0 com.onyx.easytransfer
adb shell pm disable-user --user 0 com.onyx.kime
adb shell pm disable-user --user 0 com.onyx.android.ksync
adb shell pm disable-user --user 0 com.onyx.calculator


The commands above should be enough to remove all bloatware, and if you wish to learn more, this article and this article provide a good overview.

Using an external keyboard

Given that this device is meant to be used in bed, and since it is quite unwieldy to write on the on-screen display, I was looking for an external keyboard that could be somehow attached to the BOOX, that would bestable, low-maintenance and allowed for flexibility on the writing position.

From the keyboards and keyboard stands I’ve researched, the most elegant solution I found was to use Logitech K480, a bluetooth keyboard with an in-built cradle, detachable batteries (i.e. zero issues related internal battery becoming faulty), incredible battery span (some users reported this to be in the order of years), and wonderful typing experience.

One caveat is that the BOOX slides backward and falls when using the cradle without any other aid. A very simple fix is to get a piece of cardboard, and place it behind the BOOX (see picture below) to create a better fit. It works remarkably well.


Writing and Drawing

Writing

Writing is my main use case for the device, and even though it has some lag when doing it, it works great for me, because I tend to write long streams of ranting text, rather than editing it.


The best mode I’ve found for writing was the “A2” Mode. It has ghosting, but is responsive enough for this use case.


Drawing

To draw, I use the default Onyx’s Sketch app due to its responsiveness, even though I would have preferred to have a more customizable UI interface and faster ways to share the drawing.

Sharing is one of the downsides of the Onyx Sketch app. You need to either use their cloud service, or export the image in order to get the sketch easily accessible on other devices. I’ve opted to use the export functionality, which allows to directly share the image to a Google Keep note, which fits well into my workflow. I would prefer to not have this extra step, but still, it works well enough.


Closing Thoughts

This has been one of the most satisfying devices I’ve added to my routines, and it’s incredibly satisfying to write on this device, almost distraction free. It allows enough customizability for it to be tailored to my use case, but is also limited enough to deter me from falling prey to distractions.

I hope this was useful towards your journey - do let me know your tips for distraction free environments!



  1. The BOOX Note Air3 B/W is a very capable device, able to do most of the things one would expect from an Android Tablet, but its screen refresh rate, speed and screen drag make it less suited for fast browsing and video usage for example. ↩

  2. The screenshot is from BOOX Tablet Note Air 3 C, the color counterpart of Onyx BOOX Note Air3 B/W, but the layout is the same, save from the color display ↩

]]> 2024-03-25T00:00:00+00:00 Musk's biography: why it's interesting https://lopespm.com/notes/2024/02/18/elon_musk_biography.html https://lopespm.com/notes/2024/02/18/elon_musk_biography.html I’ve been hearing the latest Elon Musk biography 1 on audiobook, and just like Ashlee Vance’s book1, I’m finding it incredibly interesting. Here’s why:

Possibilities, Monty Python, costs, whys and whos

The action on potential

I find his path inspiring. It makes me want to create and do things, and shows that many things that are thought to be impossible, are actually achievable. There is a whole realm of opportunities waiting to be grabbed, and his life demonstrates just that.

There is a cost though. If it were that easy, everyone would do it.

The defining characteristics

The book goes deeper into the unique aspects that allow Musk to be an extreme overachiever, but also the personal and professional costs those same aspects inflict.

Musk’s capacity to take pain, risk, stress and adversity is staggering. I’ve been reading several biographies over the past years and met quite a few people throughout my life. I can only find a handful of them that are even comparable.

The relentless pursuit, sometimes maniacal, of product quality, functional and non-wasteful design, and effective manufacturing. All in the name of a greater objective, larger than any single human being or corporation.

The relationships

The book dives into the relationships that shaped Musk (for the better, for the worse, or both), the types of relationships he consistently attracted (which resonated with his personality and upbringing), and essential relationships that allowed him and his projects to thrive.

They come to further prove that no man or woman is an island. You can’t succeed by your own.

The humor

Consistently throughout Musk’s life, there is a balance between dark and light, between lighthearted comedy and downright hell.

I’ve had a good laugh with some of the humorous sides though, such as when eels and hovercrafts were catered to one of his marriages, or when Blue Origin filed a formal protest against SpaceX to prevent the company from having exclusive use of a NASA launchpad, to which Musk replied that “if they do somehow show up in the next 5 years with a vehicle qualified to NASA’s human rating standards that can dock with the Space Station, which is what Pad 39A is meant to do, we will gladly accommodate their needs. Frankly, I think we are more likely to discover unicorns dancing in the flame duct.”

Shortly after, a SpaceX employee bought dozens of inflatable unicorns and took a picture of them standing in a flame duct 🦄

Musk’s algorithm

“The Algorithm”, is a distillation of lessons learned while relentlessly increasing production capacity, which Musk repeatedly preached on his enterprises:

  1. Question every requirement
  2. Delete any part of the process you can
  3. Simplify and optimize
  4. Accelerate cycle time
  5. Automate

1. Questioning requirements

It can be summarized into this: “The only rules are the ones dictated by the laws of physics. Everything else is a recommendation.”. It’s a recurrent theme.

Doesn’t come without a cost though: it is often associated with conflict and chaos.

About 2. Delete any part of the process you can

“Delete, delete, delete”

Optimization many times goes hand in hand with deletion. Deleting is hard. It requires letting go of past achievements, comfort, and acceptance that something is not coming back. Almost like a breakup.

3. Simplify and optimize

Goes hand in hand with deletion.

4. Accelerate cycle time

This is where intensity and being “hard core” are most leveraged, in my view. Again, with their pros and cons.

5. Automate

A more obvious one. Put a machine to do it. Easier said than done though.

Data points -> cause & effect probabilities

I’m fond of biographies because they provide several data points that I can later use in my life. This is, which set of actions preceded a given consequence. Of course the context matters, and surely no same action will lead to the same consequence, specially if made by different people.

But patterns start to emerge, and their respective probabilities. These help me solidify my own personal theories and strategies. Learning from others is important. It allows for “shortcuts” similar to the ones provided by good mentors.

To build these, I need data points, a lot of them.

That is why I like Isaacson’s biographies so much 2. This is this third book I read (or hear) from Walter Isaacson, and one of the things I enjoy most about his writing is how deep he goes into the details of someone’s story. I believe that details are important when portraying someone’s life. Sometimes, the smallest of events make an immense difference over someone’s path.

Just read / hear it

Love him, or hate him, I recommend Musk’s biography to just about anyone.



  1. This link is an affiliate link, and as an Amazon Associate I earn from qualifying purchases whose commissions help this small establishment, at no additional cost to you↩

  2. Except Steve Jobs Biography 1, which I could not bear to finish. Jobs personality was a bit too much for me to handle↩

]]> 2024-02-18T00:00:00+00:00 Project Planning Framework https://lopespm.com/notes/2024/01/27/project_planning_framework.html https://lopespm.com/notes/2024/01/27/project_planning_framework.html Different goals require different roadmapping and planning techniques.

In this post, a simple framework is proposed to classify a goal using three axes, in order to inform the approach taken towards reaching that goal. The three axes are:

  • Timeline sensitivity: how much leeway is there to extend the deadline to deliver a goal?
  • Dependency level between goals / projects: how many steps are needed to reach the goal?
  • Uncertainty of which steps will drive the goal forward: how sure are we that a given step will move us toward the goal?

Example of how to apply the Framework

Scenario 1: Reach Compliance

Goaling Statement

An external entity (let’s say, government or partner company), has made a set of requirements for which your team must provide compliance for on a fixed, non-negotiable timeline, with limited team capacity. The “definition of done” is well defined by this entity, and there is a low amount of unknowns of how to reach the goals.

Axes

  • Timeline sensitivity: High
  • Dependency level between goals / projects: Low
  • Uncertainty of which steps will drive the goal forward: Low

Approach

This goal has sensitive timelines and is archetypal top-down. The deadline will be reached in one way or the other, so all hands are needed on deck to tackle this goal. In specific:

  • Use a Gant-style chart, where there is a heavy emphasis on the timeline
  • Apply stricter project management, more process driven

Scenario 2: Increase Effectiveness of a User Flow

Goaling Statement

Your team supports a user flow, such as a check-out flow, for which the goal is the increase the conversion amount by X. There are known levers you have available to drive this metric, but also many others which are unbeknownst and would need experimentation and research. This increase is not business critical, so there is an underlying lenience in case the goal is not hit.

Axes

  • Timeline sensitivity: Low
  • Dependency level between goals / projects: Low
  • Uncertainty of which steps will drive the goal forward: High

Approach

Unlike Scenario 1, which is exploitation heavy, this scenario favors an approach which is exploration heavy, due to the high amount of uncertainty of which projects will be able to move the needle towards the successful completion of the goal. To de-risk and address this goal, a high level of adaptability will be needed, and several opportunities will need to be created to discover which levers can be used. In specific:

  • Have open ended brainstorming sessions
  • Leverage user research and focus groups
  • Apply looser project management, to allow for quick and less trashier pivots

Scenario 3: Unification of Several Systems into One

Goaling Statement

Your team is tasked to unify several internal systems into a single one, where several dependencies between different projects exist, and the time each project will take ranges from certain to unknown.

Axes

  • Timeline sensitivity: Low
  • Dependency level between goals / projects: High
  • Uncertainty of which steps will drive the goal forward: Low / Medium

Approach

The focus here should be on making clear which are the dependencies between different projects, and allow some leniency towards the completion of each of the projects, while actively managing the team’s capacity to focus on the projects which are discovered to require a heavier lift. In specific:

  • Build a full dependency graph, allowing for each of the projects timelines to have some leeway where needed
  • Incentivize structured system design documents and discussions, making sure that all involved parties are aligned on a given approach, and engineers are able to explain not only the system they own, but also other systems involved in their unification

Closing Thoughts

The framework above is quite simple and could potentially be used for several other different use cases. It might also be missing some essential parameters that are relevant for your specific context. Let me know if you find these, I would be happy to review the framework accordingly. Regardless, hope the above can be useful on your next roadmapping / planning cycle!

]]> 2024-01-27T00:00:00+00:00 Make the meme transcend its host https://lopespm.com/notes/2023/12/31/meme-transcending-host.html https://lopespm.com/notes/2023/12/31/meme-transcending-host.html There is an ephemeral nature to everything. That includes me, you and this blog. This blog is currently hosted in GitHub, and nothing guarantees that after I’m long gone, or even before, GitHub might cease to exist. How to account for that?

One way to keep these writings alive, would be to assign a third person with the task of keeping this establishment online. What happens when that person is gone though? It’s turtles all the way down.

Another way would be to find a self sustainable digital mechanism to keep this blog alive, ie, a bot. That has its own issues as well, since technologies change and distributed systems like IPFS come and go.

The other option to extend the lifespan of these digital contents is to have these writings persisted into a physical medium. A book. No maintenance required. As long as it can be kept safe from unfortunate book burning events, the probability that a single copy remains alive, and can be made accessible (such as a public library) to someone and hopefully improve their life, increases.

No Input, No Output

The jump from a humble blog to a book is quite a leap. Outrageous even. Perhaps doable.

Some of the books I’ve recently read gave me some guidance and ideas on how to take that germinal idea into fruition (note that the following links are affiliate links, and as an Amazon Associate I earn from qualifying purchases whose commissions help this small establishment, at no additional cost to you):

  • Rolf Dobelli’s The Art of the Good Life is a collection of 52 articles that Dobelli wrote for a newspaper column.
  • Dobelli also previously wrote The Art of Thinking Clearly (I did not read this one), which reveals the most common errors of judgment, and how to avoid them, throughout 99 short chapters. Although the idea the idea of 99 chapters seems to have been original to Nassim Taleb’s project to publish “99 (convex) heuristics”
  • Steven Bartlett shares on his Diary of a CEO book that at 21 years old, he made a promise to himself that every day at 7PM, he would write a tweet or make a video delivering a single idea, and then post it online at 8PM, which Steven claims to have been the habit the most difference to advance his knowledge and skills. It completely changed his life trajectory, and consequently it’s the piece of advice Steven most strongly urges for anyone looking to be a better thinker, speaker, writer, or content creator. The key here was that he made it a daily obligation, not just an interest.
    • From this, I assume that this habit was foundational to his Diary of a CEO podcast, which eventually resulted in the book above
  • After listening to Benjamin Franklin’s biography on audiobook, I’ve came to learn that not only was Franklin a statesman, diplomat, scientist and inventor, but also a prolific writer, printer and publisher, which were unbeknownst to me. As a publisher, Benjamin Franklin wrote several dissertations and essays 1 (such as the ones under the pen name of Silence Dogood), which I’m sure to have fed into Franklin’s books, which brought many interesting ideas to a large population, like his own autobiography, which could be seen as a compilation of several of his ideas and reflections / writings (these go hand in hand).

Apart from finding the above points inspirational, they also make me think that creating a discipline where I consistently share my reflections and experiences via writing or videos, not only helps me solidify my thoughts and provides more immediate feedback on how valuable these are to its viewers / readers (which allows me to course correct and iterate 2), but also provides the building blocks to put together a hopefully meaningful and concise book to help surpass the limited lifespan of the digital medium that hosts this blog and my YouTube videos.

No Input, No Output

― Joe Strummer, lead singer of The Clash

The Path Towards Physical Persistence

Given the above, here are some action points to put the wheels in motion towards building “physical persistence” via a book, and give better chances for its memes to transcend their host 3:

  1. My low priority 2024 goal is continuing to publish blog notes and videos. The numeric goal I’ve set for myself is to publish 12 blog notes, about once per month. From these, make the respective YouTube videos, but this would be a stretch sub-goal (albeit, one that I really enjoy)
    • This was a consequence of my recent “personal performance” review for this year’s personal goals, which was a good opportunity to reflect on what went well, what went less well, what was achieved, and what was missed. One of the aspects that I aim to implement for the upcoming year’s goals is to use numeric goals, where applicable.
  2. Once 50 or 99 meaningful blog notes are written, attempt to compile them into a book.
    • Why 50 or 99? Upon reaching 50 notes, I might feel that not enough substantial content was generated to be distilled into a book. Once 99 notes are reached however, it would be more certain that enough meaningful content was accumulated (hopefully). I would expect the distillation process to remove several notes that would be deemed to have less quality.

Looking forward to 2024! 🎉



  1. On the extensive range of writings that Benjamin Franklin did, there are also hilarious examples such as Fart Proudly, an essay about flatulence written while he was living abroad as United States Ambassador to France↩

  2. One the learnings from making the Survival Ball video game, was that the lack of exposure throughout it’s development process lead to less visibility to the ones who might be interested in it, and less immediate feedback on what worked and what didn’t work, which could have possibly have helped to craft an better product, by virtue of allowing faster / less painful pivots↩

  3. Coined by the British evolutionist Richard Dawkins in his book The Selfish Gene (1976), a meme is a unit of culture—such as “tunes, ideas, catch‐phrases, clothes fashions, ways of making pots or building arches.” In humans, memes have supposedly taken over much of the evolutionary burden of the traditional units of heredity, the genes. Dawkins introduces them because in his opinion the rate of human cultural evolution is far too rapid to be simply a function of gene‐centered evolution.↩

]]> 2023-12-31T00:00:00+00:00 New York City Guide https://lopespm.com/notes/2023/12/25/nyc.html https://lopespm.com/notes/2023/12/25/nyc.html

I was recently in NYC. It was one of the most interesting experiences I’ve ever had.

Each one has their own perspectives when visiting the city, and I’ve asked several people about theirs before the trip, which helped immensely when planning out my itinerary. Now it’s my time to pay it forward. Here is my guide / personal perspective of New York.

Maps

Having a pre-curated list of points of interest on Google Maps was one of the most useful and time efficient aspects of the entire trip. This allowed me to quickly improvise and make the most of my surroundings, since I could just look at the map to check which nearby places I could visit at any given time.

Museums

I didn’t consider myself to be an excessive photographer. I’ve discovered on this trip that I was probably wrong. I can’t recall the last time I took so many photos, it was like I was in an outstanding candy store of visual goodies.

Intrepid Museum

Let’s start with fashion. I’ve been pointed out that my glasses make me look nerdy / square / too serious / too much like an engineer. Well, thanks to this museum piece, I can now say they were developed for space walks:

Oh, and look back and you will also see the Space Shuttle Enterprise, the first orbiter of the Space Shuttle system:

You can find these glasses, the space shuttle, sit inside the Mercury capsule or A-6 Intruder cockpit, see real life jet planes like the Lockheed A-12, in the Intrepid museum.

Where is the Intrepid museum? The museum is an World War II–era aircraft carrier. Yes, the vessel is the museum. You can explore several of it’s interior sections, including the bridge, living quarters, gun and bomb bays, go through the same narrow escalator that pilots took to the flight deck, all of these throughout several floors and different explorable rooms

Right beside the Intrepid lies USS Growler, a diesel powered submarine retrofitted to deliver Regulus nuclear missiles, as a form of nuclear deterrence during Cold War. You can go inside the submarine and get an intimate experience of how it’s lived throughout their 2 month patrols, where they faced the constant scenario of being called upon to launch a nuclear missile onto Russian soil, destroying not only their target, but most likely kill themselves on the process, since the process to launching / reloading Regulus missiles was long and would expose this submarine to enemy reconnaissance.

  • Since it was a diesel submarine, it required to be at surface or near surface level (it was not possible to push out their exhaust gasses when deep in the ocean) to enable a set of its 3 engines to charge its on-board batteries that would power the submarine and its electric motors.
  • It also needed to be periodically refueled, so the submarine ran a long circular patrol in the Pacific Ocean to refuel to eventually sit near Russian enemy lines for dozens of days.
  • The crew had to maintain absolute secrecy about their mission, and none of their partners knew the context of it
  • The families of the crew members knew nothing about their mission, and that if they did launch a nuclear missile, they would be exposed, and most likely be killed as they expose themselves to launch the nuclear load (not to mention the annihilation that comes from the nuclear warhead)

I found the underlying story and details of this submarine and this mission to be so interesting, that right after the tour, I went immediately in search of staff members to ask a series of questions that came about during the visit. They very kindly and patiently took the time to answer them, and one of the ladies even asked me if I was an engineer. I’m not sure what gave that away. It was either the questions, or my glasses :)

MET

MET, or Metropolitan Museum of Art, has two locations: the museum in Fifth Avenue is the most famous one and is often referred to when mentioning MET. The other lesser known branch is in Upper Manhattan, the MET Cloisters.

MET: Fifth Avenue

  • If you like the British Museum, you will love MET on Fifth Avenue
  • They reconstructed the 10 BCE Temple of Dendur Roman Egyptian religious structure, which was originally located in Tuzis (later Dendur). Must I say more?
  • This massive museum spans through classical antiquity and ancient Egypt; paintings and sculptures from nearly all the European masters; and an extensive collection of American and modern art. All of these are intertwined with large thematic rooms where the installations are beautiful to behold. If you are going to see a single museum in New York, this should be the one.

One of the last books I’ve listen on audiobook was Walter Isaacson’s Benjamin Franklin biography, where one of passages goes through the story of how a Duplessis’ painting of Franklyn’s portrait came to be. This 1778 painting can be seen in the museum, and curiously enough, Franklyn’s depiction on $100 bills from 1914 to 1990 had him wearing a fur coat, just like this painting.

MET: Cloisters

With the same ticket, one can visit the Cloisters and the 5th Avenue museum on the same day. The Cloisters museum is America’s only museum dedicated exclusively to the art and architecture of the Middle Ages, and is smaller than the 5th Avenue one. Still there much to explore not only inside the museum, but also on its surroundings

Surrounding the Cloisters is the beautiful Fort Tryon Park and it’s Heather Garden, sitting right next to the Hudson Riven. A bliss to behold.

National History Museum

  • I lost count of the amount of dinosaur skeletons I’ve seen in the natural history museum. And nothing quite prepares you to see a gigantic skeleton of a Titanosaur. It’s so big that its head extends outside of its new home in the Museum’s fourth-floor gallery.
  • I’ve learned while browsing the museum about mass dinosaur graves and their relationship to draughts, and went through thought provoking sections, such as the presentation of a massive turtle specimen, that if it were to be extinct it would probably be right part of our imaginary and event movies.

Another thought that stuck to me is how deeply connected we are to other animals, Earth and outer space. Seeing life sized representations of several of these makes them immediately more relatable.

Guggenheim

On Saturdays, from 5 to 8 pm, admission to the museum is “Pay What You Wish”, for a minimum of $1. I would recommend doing so, since the museum can be easily visited in less than one hour, and in my opinion, it’s more about the building than the pieces themselves

Entertainment

Broadway

Broadway shows are a hallmark of NYC, although if you are from London or Europe, I would recommend not seeing shows that are already available in London. For example, I’ve heard that Hamilton is better seen in London, due to the larger contextual part that King George has in the UK.

The Chicago musical is a good bet, and is the one I’ve attended to, it is the second longest-running show ever to run on Broadway, behind only The Phantom of the Opera

Comedy

Comedy Cellar is a comedy club in Manhattan where many top New York comedians perform, and where several comedians started, like Louis C.K. or Dave Chappelle. I found it be have down to earth, edgy comedy, and you are asked to leave your phone in a pouch, which I believe to increase presence of the entire crowd, but also provides extra freedom to comedians, given to how powerful cancel culture can be

Points of Interest

Statue of Liberty

You can get the Staten Island Ferry from the Whitehall terminal, for free. On your way to the terminal, several people will try to sell and cajole you into payed trips to the Statue of Liberty trips. The ferry is more than enough.

  • The ferry runs every 30 minutes, 24 hours per day, for most of the days, and travel time is approximately 25 minutes.
  • Once you get to Staten Island, you can either take some time to explore it, or just return immediately to Manhattan, by exiting the ferry, by entering St. George’s Ferry Terminal, and entering the boarding gates for the next ferry.
  • When entering the Whitehall terminal, enter through the right side of the ferry in order to be on the same side that the Statue of Liberty will appear. If entering on St. George’s Ferry Terminal, enter through the left side of the ferry. Regardless, you can easily change sides when inside the ferry
  • Most of the people will likely go to the top floor with larger windows, which will eventually get crowded. I recommend instead to take the bottom floor, and find a window that is already opened. The view is much clearer (since you don’t have a worn down piece of glass obfuscating the view), you can sit comfortably near the window, and you won’t need to fight a crowd (in principle)

Central Park

Central Park is an urban park between the Upper West Side and Upper East Side neighborhoods of Manhattan in New York City that was the first landscaped park in the United States. It is the sixth-largest park in the city, and it’s a great way to step out of the busyness of the city and discover the treasures that lie in the park.

High Line and Little Island

The High Line is a 1.45-mile-long elevated linear park, greenway and rail trail created on a former New York Central Railroad spur on the west side of Manhattan in New York City. If starting it from the north side, you can enter it near Hudson Yards, and walk your way south from there.

Upon the southern end of the High Line park, you can view Little Island, and quickly walk to it. Little Island is an artificial island park, with some beautiful views and interesting layout

Financial District

9/11 Memorial

The memorial is located at the World Trade Center site, the former location of the Twin Towers that were destroyed during the September 11 attacks. Each of the towers’ footprints are now the home of two large, recessed pools. The sheer dimension of these is awestrucking. The entire site is filled with symbolism.

Wall Street

  • On the street itself, we can find the New York Stock Exchange, the largest stock exchange in the world, which for decades filled the popular imagination with its hustle, bustle and hectic traders. Those same brokers and traders are now surrounded by computers that manage the majority of the buying and selling of stocks for their various accounts. Floor trading still exists, but it is responsible for a rapidly diminishing share of market activity.
  • The Charging Bull of Wall Street is not really located on Wall Street itself, but it is quite near. The bull in finance represents optimism and growth. The statue on Wall Street represents the same ideas.
  • The Federal Hall is a historic building, whose original building served as New York’s first City Hall and hosted the 1765 Stamp Act Congress before the American Revolution. With the establishment of the United States federal government in 1789, it was renamed Federal Hall, as it hosted the 1st Congress and was where George Washington was sworn in as the nation’s first president. The current structure was built as the U.S. Custom House for the Port of New York before serving as a Subtreasury building from 1862 to 1925, and the current national memorial commemorates the historic events that occurred at the previous structure.

Columbia University + Morningside Park

Walking west from Central Harlem, where you can find the Apollo Theatre for example, we reach Morningside Park, from which we can climb up towards Columbia university, a private Ivy League research university in New York City.

Grant’s Tomb

  • In a mere 7 minute walk from Columbia University, you can reach the General Grant National Memorial (Grant’s Tomb), classical domed mausoleum, the resting place of an American military officer and politician who served as the 18th president of the United States from 1869 to 1877. The underlying background music and the attention to detail are something to behold
  • Right beside it sits the soaring Riverside Church, all of this surrounded by the peaceful Riverside Park

Citigroup Center

Due to material changes during construction, the building as initially completed was structurally unsound. To save money, Bethlehem Steel changed the plans in 1974 to use bolted joints, and wind loads were calculated from perpendicular winds, as required under the building code; in typical buildings, loads from quartering winds at the corners would be less.

In June 1978, after an inquiry from an engineering student, the structural engineer recalculated the wind loads on the building with quartering winds, and found these to significantly increase the load at the bolted joints, and that a wind capable of toppling Citicorp Center would occur every 55 years on average.

Starting in August 1978, construction crews covertly fixed the issue, and six weeks into the work, a major storm was off Cape Hatteras and heading for New York. The reinforcement was only half-finished, with New York City hours away from emergency evacuation. The storm eventually turned eastward and veered out to sea. The repairs were finished successfully, and no major issues happened ever since.

The Woolworth Building

Completed in 1912, the Woolworth Building was the tallest building in the world from 1913 to 1930, with a height of 241 meters. It blows my mind that such a tall building was built more than 100 years ago.

Billionaires’ Row

Home of one of the most expensive apartments in the world, the Billionaires’ Row hosts a group of ultra-luxury residential skyscrapers. Several of them are so thin that it’s hard to believe how they can stand upright without toppling over.

Lincoln Center

  • The Lincoln Center is a complex of buildings in the Lincoln Square neighborhood on the Upper West Side of Manhattan, where the beautiful Metropolitan Opera House can found.
  • It houses internationally renowned performing arts organizations including the New York Philharmonic, the Metropolitan Opera, the New York City Ballet, the Chamber Music Society of Lincoln Center, and the Juilliard School.

Grand Central Terminal

  • Grand Central Terminal (aka Grand Central Station or Grand Central), is a commuter rail terminal located right in the center of the city. So even if you don’t stumble upon it during your commute, it can still be easily reachable via public transportation.
  • It has been the subject, inspiration, or setting for literature, television and radio episodes, and films, and every year the MTA hosts about 25 large-scale and hundreds of smaller or amateur film and television productions.

The Bronx

The Joker Stairs

The first picture on this note was taken at the “Joker Stairs”, which is the colloquial name for a step street connecting Shakespeare and Anderson avenues at West 167th Street in the Highbridge neighborhood. The stairs are quite different from when Joaquin Phoenix danced as the Joker. They are more colorful, and have a construction site ongoing at the top.

The Birthplace of Hip Hop

DJ Kool Herc is credited with helping to start hip hop and rap music at a house concert at 1520 Sedgwick Avenue, which is currently covered by scaffolds as seen here. I would recommend to go during daylight hours and to be attentive to your route, as it gets rougher on the way there.

Yankee Stadium

The stadium is the home field for the New York Yankees and New York City FC, and also has some small surrounding parks.

Brooklyn

Williamsburg (Bridge)

While the Brooklyn Bridge stands as the most famous bridge to cross the East River, you can easily dodge its flocking crowds by crossing the Williamsburg Bridge, which leads you into Williamsbridge, characterized by a contemporary art scene, hipster culture, and vibrant nightlife that has projected its image internationally as a “Little Berlin”

You can find several interesting places in Williamsburg such as the Domino Park, Spoonbill & Sugartown Books and the City Reliquary Museum.

Brooklyn Bridge

The Brooklyn Bridge is an iconic landmark of NYC, which you can cross from Manhattan towards Brooklyn. Be advised that this crossing is often quite crowded, and there are several vendors in the bridge itself

Dumbo & Brooklyn Heights

Once you cross the Brooklyn Bridge, you can head directly to Dumbo, where you have a beautiful view to the Manhattan Bridge

You can also have several scenic views towards of the Brooklyn Bridge and Manhattan, in both Dumbo and Brooklyn Heights

Brooklyn Museum & Brooklyn Library

  • The Brooklyn Museum is New York City’s second largest and contains an art collection with around 500,000 objects, and is fairly easy to access from the subway. The breadth and depth of objects at the Brooklyn Museum is best compared to the Met, boasting strong departments of both Western and non-Western holdings. I didn’t fit enough time to visit the actual museum, but it is definitely on the todo list in case I visit the city again.
  • Brooklyn’s Library four stories high main entrance is a marvel to behold just by itself

Prospect Park

Prospect Park is an expansive and peaceful urban park with beautiful landmarks and structures, such as the Boathouse on the Lullwater, several watercourses, bridges, monuments and statues

5th Avenue

Rockefeller Center Christmas Tree

The Rockefeller Center Christmas Tree has been a yearly tradition ever since 1931, which now hosts a 20m+ Norway spruce, near to an also historical skating rink, which was opened below the tree in the plaza in 1936. This year, you’ll be able to see the tree up until January 13th 2024.

Saks

  • Directly facing the Rockefeller Center, you can find Saks Fifth Avenue, an American luxury department store founded in 1867.
  • In partnership with Dior, as part of Saks’ annual holiday, it is hosting a massive, bronze circle is broken up into eight sections with a deep blue center decorated with all the zodiac signs and a spray of stars, with one big star in the middle, where there is a periodic visual and sound show which is truly impressive.

Trump Tower

Just a few blocks north from the above places lies the Trump Tower, where Donald Trump descended on an escalator to announce his candidacy for president, the first step on a journey few believed would take him all the way to the White House.

Eat, Sleep, Move and Pay

Where to Eat

  • Unless you would be aiming to hit a specific high demand restaurant at a specific time span, there is limited need to pre-book a restaurant, since you can just walk for a few minutes and find other quality restaurants.
  • There are several food stalls around NYC, many of which you will be able to recognize by their LED letterings and smell of burnt meat, I would personally avoid these. There are food stalls which are incredible though. One of these is Shawarma Bay, just a block away from Radio City Music Hall. Incredible food, although it can gain quite a queue, which can take about 30 minutes, as it was my case. Worth the wait though– cops were eating there
  • You can get incredibly good Deli Sandwiches in NYC, in places such as Sunny & Annie’s Deli. I’ve personally tried one at Gold Deli, and it was seriously good.
  • Are you in Times Square and want to grab a bite? Walk 2 blocks west and you will find yourself in Hell’s Kitchen, packed with good restaurants, without much less hassle
  • Wan Wan Thai restaurant was the best dining experience I had in NYC, for an acceptable price.
  • Joe’s Pizza offers traditional New York slice-style pizza at a fair price, that has made a dent in popular culture. Definitely worth the it.
  • Want to grab quick, healthy, tasty bite? Dig Inn has you covered.
  • Williamsburg’s L’Industrie Pizzeria apparently has some of the best pizzas in NYC, but I can’t attest to that, given there was a massive queue when I arrived there. So I moved to Rosa’s Pizza, where I had delicious slice.
  • On markets such as the Chelsea Market you can find several good food places.
  • Fraunces Tavern is a museum and restaurant that played a prominent role in history before, during, and after the American Revolution. At various points in its history, Fraunces Tavern served as a headquarters for George Washington, a venue for peace negotiations with the British, and housing federal offices in the Early Republic.

Where to Stay

  • This all depends on your budget, activities you would like to do, who is going with you, etc. But in general, I would advise on staying in Manhattan island, or at least near to a transport connection that can get you quickly to the city center, since it is where most of the points of interest are located, and transport links are quite good from there. Most likely you will also be in your lodging to sleep, and not much else.
  • If looking for a hostel, HI New York City Hostel is a good one. It even sports free weekly comedy sessions every Sunday night. It’s located near Central Park, and you can’t go wrong with this one. In general, you can search through https://www.hostelworld.com/ and book through there, it was quite reliable for the two bookings I did.
  • If looking for something more comfortable, citizenM Hotels are of incredible good value and location. Apart from the comfortable rooms that have nice view already, on Bowery it even has rooftop terrace with an amazing view of the city

Money / How to Pay

  • Stores, food establishments (even chains), bodegas / delis, 7-11s, etc present their prices as pre-tax. New York City sales tax ranges from 4% to 8.875%, and some products are exempt from it.
  • Be prepared to tip everywhere. On restaurants it is customary to tip 20%. There is some debate about whether you should calculate the tip on the amount before or after tax.
  • You don’t really need cash. You do need it if you want to buy from street vendors, or if you want to dodge the additional cost that certain sellers impose on card transactions (like the shawarma above)
  • On restaurants and entertainment venues, it is common practice for the attendant to leave with the credit card, to later return with an attached invoice that is meant to be signed and filled with the tipping amount.
    • It also happened to me that when trying to pay by contactless card to a market vendor, the card kept being rejected, and since the card insertion feature was not available, the vendor wrote down the card number and details into the system to proceed with the payment. That failed as well, because the card got blocked by my bank due to a false-positive suspected fraud, as I later discovered. Eventually I just payed with cash to resolve the issue.
    • As an European, the practice of being physically separated from the card, verification via signature, and freely writing down credit card numbers strike me as quite unsecure.

Getting Around / Transportation

  • Walking is one of the best ways to discover the city. Sidewalks are broad, the city is grid shaped (so expect to be given coordinates in the form of street / avenue to pinpoint a place), and pedestrian lights and timings ran standard everywhere I visited. The large majority of my city exploration was made over the span of 180km, throughout 4 days. Extra tip: if you are used to barefoot shoes, wearing firm ground outsole Vivos to do so is amazing. My feet were sore, but happy.
  • Walking through the perpendicular numbered streets and avenues is an absolute bliss, and makes it quite easy to navigate through them. Several times I just looked at the map and quickly memorized something like “3 right, 2 left”, which translated to “cross 3 roads and turn right, and then cross 2 roads and turn left”. This would keep me walking going for a long time and have time to appreciate the surroundings, without the need to constantly check the map.
  • Ferry boats are a great way to sight the city, for a low price ($4 for a one way-ticket), and you can buy these tickets through the NYC Ferry app or a ticket vending machine.
  • Subway and Bus now have OMNI, which just means that you can tap in with your credit / debit card. Each ride costs $2.90, and there are no different prices for different zones. This has a [rolling 7 day price cap] (https://omny.info/fares), so you never pay more than $34 over that period.
    • Metro Card is the pre-paid version of this. If you need one, buy it directly from the machines to avoid being scammed. For example, a 7-Day Unlimited MetroCard costs $34 upfront, and can be used any number of times during those 7 days.
      • Be aware that if you swipe in, then get out from the station, and then attempt to shortly swipe in again, it will reject your swipe, due to being too recent. If that happens, ask a MTA attendant to let you in (if available), or do like many others and wait for someone to crack open the emergency exit from the inside. If you hear an alarm going off in the station, most likely it is because someone opened the emergency exit to let others in.
  • Just like routine events such as figuring out post-tax prices purchases or how much and when to tip require some kind of mental effort, figuring out the different edge cases of the subway system will keep your mind busy. It is like there is a perpetual rat race game being played in the city.
    • Sometimes trains stop at all stations, sometimes they don’t, sometimes trains have indicators of which station you are in, sometimes they don’t, sometimes the station has (non-standard) indications of how much time it takes for the next train to arrive, sometimes they don’t. Sometimes it rains heavily inside the stations, sometimes it doesn’t.

Other Notes

  • Something unexpected to me, which cannot be transmitted through visual media, is the collection of smells around New York, in almost all parts of the city, except a set of parks and neighborhoods mostly outside Manhattan. The smell of garbage (since it is comment that it accumulates in heaps of bags on city sidewalks), burnt street meat, horse manure in some parts of Central Park, a smell I can only describe as pervasive detergent / laundry smell that could even be detected inside several buildings like the MET and Delis; the strong smell of road paint (since it appears that several of the streets were recently repainted), and signature smells from the subway
  • Several subway lines are relatively shallow, so they are easily heard at street level. They also generally run below the roads themselves, which I would assume to make it easier when building incredibly high skyscrapers whose foundations run quite deep.
  • Be attentive of your surroundings and where you walk, such dodging through garbage debris and sidewalk cellar doors, be them open or closed
  • Be prepared to encounter people who are not as well off, and might be living in the streets. Use your judgment, keep out of confusions, be respectable and don’t engage with people who have serious mental issues. If you are in a rougher area, memorize your path when possible, and stop at a local shop if you need to get your bearings. Use your common sense
  • You might have noticed that I didn’t mention the common Edge / Top of the Rock / Summit One / Empire State Building / etc observation decks. It’s not that I don’t enjoy incredible views, but it didn’t fall high enough on the list of priorities, specially because I’ve had the opportunity to see skyline views through other buildings I’ve passed by, such as the Hotel.
  • I tend to like long trans-atlantic flights, since they provide a moment of low stimuli, where there is less incentive to use the internet. My personal piece of advice would be to avoid the usual in-flight entertainment. I love to use them to resume a book or to write down my reflections from a trip for example (several of article lines were written during the return flight)
  • This only shows my ignorance, but I had no idea that so many well-known artists were originally from New York, like Jay-Z (Brooklyn), Cardi B (South Bronx), Spike Lee (Brooklyn), and of course, Jenny from the block. That explains why I’ve consecutively heard “Empire State of Mind”, from Jay-Z and Alicia Keys (Hell’s Kitchen), and “All I want for Christmas is You”, from Mariah Carey (raised in Huntington).

Closing Thoughts

Ever since I was a kid I had a considerable fascination with the USA. The culture that arrived through the small and big screens, the big companies and entrepreneurs that I admired, the language, the people I’ve met along the way, the stories. Having had the opportunity to visit and experience it first hand was a true privilege. I’ve lived it like it was my last time, and it imprinted in me several thoughts and perspectives that I’m sure to last a lifetime.

New York ain’t no DisneyLand, and Elmo’s pictures in Times Square don’t come for free either, so set expectations accordingly. It can be one of the most fulfilling experiences you’ll ever have. I advise you to come prepared and plan accordingly.

Frank Sinatra sang that he wants “to wake up in the city that doesn’t sleep”. That phrase has a deep meaning that only struck me during my trip. The city does not indeed sleep. The Times Square lights, 24 hour subway, the constant movement. Living in NYC wakes you up, not only literally, but also figuratively. It wakes you up to a different reality that keeps you on your toes in several ways. In a way, it’s a celebration of life.

]]> 2023-12-25T00:00:00+00:00 Job Safety in the AI Era https://lopespm.com/notes/2023/12/25/job_safety_in_ai_era.html https://lopespm.com/notes/2023/12/25/job_safety_in_ai_era.html Words are cheap and keep getting cheaper. The last notes on this blog could have been written via generative AI, and as time goes by, new generative models will likely create higher quality content than I could ever produce, consistently.

I didn’t use these because the writing process comes as a form of thinking, but if there are automated mechanisms that increasingly outperform us humans in so many different areas, what does that leave us?

Human connection

How comfortable would you be in having a relationship with a being that is orders of magnitude more intelligent and capable than you? Not just logical-mathematical intelligence, but emotional, linguistic, musical or even spatial intelligence.

A being so advanced that any form of communication or relationship between you and it would always be severely constrained, even with neural interfaces. A being so advanced that you would never be able to have an essential understanding of it. A being so advanced that you wouldn’t even know that you are being manipulated by it. How many humans do you know who are friends with chimpanzees or bonobos, our closest relatives?

We are still human beings, with all the perks and limitations that come with it.

I would claim that humans will always be attracted to their familiar counterparts. Other beings that share their own struggles and limitations, and by consequence, what they create and share. Machines have their important place in the world, and history has proven to us that life as whole improved as technology evolved to serve us, but it would be healthier for us to think of them as tools, rather than human replacements.

Bioengineered

We have been fitted with extraordinary senses and tools that allow us to efficiently interact with the world. It took milenea for nature to shape us, and as Leonardo da Vinci puts it when comparing natural to artificial structures: “Nature made the most perfect inventions. There is nothing superfluous, there is nothing lacking” (paraphrased)

Our place

Try having a machine fixing the leaky pipe in your bathroom, or to change your desk’s lightbulb. Odds are that you’ll be flooded and in the dark in the nearterm. There will come a point where they will likely do that, but we are still not there.

Machines / tools currently outperform us in several tasks on the digital realm, and in several heavy duty physical tasks. But when it comes to fine digital dexterity, human connection, and interaction with the real world and its complexities, we still have an edge.

Future Proofing

Take a look at the daily tasks in your job. How repetitive are they? If they are repetíve, how much of a human element is present? Look at your organization and see where rubber meets the road in terms of human connection and physical intervention.

Strive to be closer to that human interface and focus on delivering tangible value to people or businesses (group of people). With this simple mindset, I believe you’ll be in a safer place, surrounded with an abundant supply of meaningful (monetary, emotional, etc) value that you can extract and provide.

]]> 2023-12-25T00:00:00+00:00 Relative Cost / Benefit of an Investment https://lopespm.com/notes/2023/10/21/investment_relative_cost_benefit.html https://lopespm.com/notes/2023/10/21/investment_relative_cost_benefit.html Let’s say you spend $1,000 to acquire film equipment (camera, microphones, and lights) to shoot YouTube videos. If you use this equipment to film and publish 2 videos, it means that each of those videos costs $500 to produce. If 100 videos are published, each of them cost $10. If all of these videos are able to generate $10 of revenue, then the entire cost of the initial investment was amortized.

The above is a severely gross simplification1, but the key point is that an investment left unused has a high associated cost. In the case above, if only two videos were to be produced, then it would have been a better idea to rent the equipment instead.

This simple heuristic of mapping the investment cost to its usage and upside, can be a potential rule of thumb in different scenarios.

Other Examples

  • Apartment Rent
    • You are renting an apartment for $1,000 per month. During the week you work in an office, on weekends you are sometimes out, and during the year you take sporadic vacations outside. Let’s say that, on average, you spend 14 hours per day on your apartment, including sleep → 14 hours * 30 days = 420 hours.
    • This means that it costs costs about $33 per day to live in the apartment, or $2.3 per hour.
  • Study → Work → Net Worth Gain
    • You studied 30 hours per week for 3 years (30 hours per week * 52 weeks * 3 years = 4680 hours) to land a job that increases your net worth $10.000 every year (on average), for 30 years. A total of $300,000 of networth gains.
    • This means that you will gain about $64 for each hour invested in studying, or $274 per day.
  • Clothing
    • You buy pair of trousers for $50, and wear them twice per month throughout for 2 years (24 months * 2 times per month = 48 times these were used).
    • This means that it costs about $1 to wear them each time.
      • By contrast, if these trousers were only used 7 times, it would cost about $7.14 to wear them each time.
  • Gaming
    • You buy a Playstation 5 for $499.99 and 10 games ($60 each) totalling $1.100. You play 60 hours per year during 5 years (60 hours * 5 years = 300 hours).
    • This means that each playing hour costs about $3.6.
  • Phone
    • You buy an iPhone 15 Pro for $999, and use it 4.8 hours per day during 2 years (4.8 hours * 365 days * 2 years = 3504 hours).
    • This means that it costs $0.29 per hour of active usage.
  • Computer
    • You buy a MacBook Pro 13” for $$1299, and use it 3 times per week for 2.5 hours per session, throughout 7 years (3 times per week * 2.5 hours * 52 weeks per year * 7 years = 2730 hours)
    • This means that it costs $0.48 per hour of usage.
  • Musical Instrument
    • You buy a guitar for $100. You play the guitar once per week, for one hour, during 2 years. 52 weeks * 1 hour * 2 years = 104 hours.
    • This means that each hour playing the guitar has a cost of $1.
  • Video Streaming
    • You have a standard Netflix subscription that costs $15.99 per month, and watch 3.2 hours (3.2 hours * 30 days = 96 hours per month) of videos throughout that month.
    • This mean that each hour costs about $0.17.
  • Audio Streaming
    • You have a premium Spotify subscription that costs $10.99 per month, and listen to about 25 hours of music and podcasts throughout that month.
    • This means that each hour costs about $0.44.

We could go on with further examples, as this simple heuristic could be applied to several scenarios, and the cost cost benefit analysis could have any perspective one so desires, but I would invite you to visit your storage and search for something you acquired long ago that got used less times than expected, and do a rough calculation of what was it’s relative cost / benefit. The same for any other investment you did in the past.

Was it a surprising result?

How did it stack up against your initial expectations?


  1. This gross simplification is meant to keep the example simple, and does not take into account other direct and indirect costs, such as the computer needed to edit the videos, electricity costs, office space costs, labor costs. It also does not take into account the gained equity, this is, you could sell the equipment and get some of that investment back. This was not considered because if the equipment breaks its value is rendered to zero, and when / if it is sold, it would be for a fraction at the initial value, not only because its inherent value would likely decrease through the years, but also because of inflation.↩

]]> 2023-10-21T00:00:00+01:00 Importance of Goal Clarity for Delegation https://lopespm.com/notes/2023/09/30/goal_clarity_in_delegation.html https://lopespm.com/notes/2023/09/30/goal_clarity_in_delegation.html Previously, in “Learnings as a Software Engineer Techlead”, I’ve alluded to the 75/25 rule, where if you are ~75% certain that someone will correctly execute a given project/task, then delegate. The other ~25% is the unknown space that the person needs in order to learn, explore, make errors, innovate and grow.

If the project/task was decided to be delegated, what comes next?

Clarity of Goals

One common challenge I’ve confronted when delegating was how unpredictable and uncontrollable the results felt upon handing over the work. This unstructured approach often left me thinking how I could decrease the non-completion risk by diminishing the delegated scope, and created confusion on the delegatee side on how they could proceed.

One tidbit of information from the “Who not How” book provided the guidance and structure to unlock this issue:

(…) research has found that teams who have high levels of autonomy but low clarity goals, as well as little performance feedback, perform worse than teams with low autonomy.

― Who not How, by Dan Sullivan with Dr. Benjamin Hardy

It is claimed in the above research that teams which have high autonomy, high goal clarity and regular feedback on their results, were observed to have their performance shooting through the roof.

Apart from regular feedback, goal clarity is incredibly important, and even though it might sound obvious, it can be overlooked and drowned out on the process of delegation.

Clarity, in Practice

After the above realization, I’ve made it a priority for each piece of delegated work to clearly define what was expected as an outcome, its definition of done, important milestones and deadlines, when should an issue be scaled to me, and which are the ownerships for each of the participants.

All of the above communicated in a clear and concise way, and preferably publicly documented to increase accountability and decrease ambiguity (depending on the situation, merely communicating these verbally might be enough, especially if the task is simple).

The Delegation Formula

Given the above, the proposed formula for delegation is:

Delegation = [ 75/25 rule -> Goal Clarity + Autonomy + Regular Feedback ]

]]> 2023-09-30T00:00:00+01:00 Life has several exits https://lopespm.com/notes/2023/08/19/life_exits.html https://lopespm.com/notes/2023/08/19/life_exits.html Life has several exits:

  • The exit from your workplace. The last day of work for your employer or your company.
  • The exit from that youth group that accompanied you through teenage years and early adulthood.
  • The exit from your high-school.
  • The exit from your hometown.
  • The exit from a long term relationship.
  • The exit from life. Your funeral.

Upon your exit, your contributions, actions, decisions, opinions and presence most likely will have affected the trajectories of the people and the world around you, in obvious and less obvious ways. Perhaps your own trajectory was deeply changed as well.

These exits may be celebrated, be a special day with fulfilling words and thoughts. They may be the last time you see a set of people. The last time you will see them in your entire life.

Perhaps upon your final exit, you have made such consequential dents on the world that your name will be engraved into the annals of history for centuries to come. On the other hand, contemporary personalities such Steve Jobs, Bill Gates, Elon Musk and Warren Buffet might be mostly forgotten a few centuries from now. People will move on, so most likely you should be grateful if your loved ones keep you in their thoughts for those ensuing decades.

Life has several exits. Life is also short and limited. There’ll be a finite number of exits you will experience. Make the most out of each period they enclose.

]]> 2023-08-19T00:00:00+01:00 The real question behind 'what do you want?' https://lopespm.com/notes/2023/08/05/real_question_behind_what_you_want.html https://lopespm.com/notes/2023/08/05/real_question_behind_what_you_want.html It’s easy to want something. If you are fortunate enough to be on income level 4 living in a peaceful nation, the world has so much to offer, within easy reach. Yet, the currencies required to enjoy them are limited: time, energy, health, money. Even though it is easy to make lists of everything one wants (typically occurring on yearly passages, such as new year’s and anniversaries), only a handful of them are achievable.

Smaller lists require sacrifices

Given that lengthy lists are unachievable, they need to be slimmed down to a manageable and realistic size. They need to be prioritized. Prioritization is an euphemism for sacrificing.

It’s easy to want something, it’s hard to choose it against another desirable one.

An example

At the start of each year, I produce a small list of 3 to 4 high priority goals and 1 optional low priority goal of what I want to achieve by end of it, along with a small description on why I want them, and their impact. This takes several hours, sometimes days to put together, because of all the other ones that need to be left out.

Once the list is made though, it’s incredibly powerful due to it’s easy recall. This means that whenever I am faced with conflicting preferences, I can easily remember it and know in a snap what is the right thing to do. That is the real power behind that list. It’s guiding all the tiny small steps and decisions happening throughout the year, bringing me progressivily closer to those goals.

The question

The real question behind defining knowing what we want is actually what we are willing to sacrifice. Once the latter is defined and we are 100% convinced about it, then knowing what we want is bullet proof and easy to follow through.

What are you willing to sacrifice?

]]> 2023-08-05T00:00:00+01:00 Writing as a form of thinking https://lopespm.com/notes/2023/07/02/writing-as-a-form-of-thinking.html https://lopespm.com/notes/2023/07/02/writing-as-a-form-of-thinking.html Large language models (LLM) like ChatGPT triggered a remarkable societal and computational shift that feels comparable to the impact from the initial internet era with its first impressive search engines like Google. LLMs are the toast of the town.

Writing emails, reports, school homework, analysis, code, summaries. The list goes on.

Save from when you would delegate this task to another fellow human, long hours would be passed in the past trying to conjure a piece of text that could communicate something to another entity. Now this can be made at scale, at low cost, with low effort.

Since we are still liable for the results of this tool, i.e. it’s still our name on the email sender, instead of just sending that email, we might first read that LLM output, interpret it, understand it, and then send it.

More than just text

Text is a form of communication. If something, or someone wrote it for us, certain decisions were made along the way to convey the goal that we gave. Out of the many paths possible to crystallize that piece of knowledge into a piece text, one of them was chosen.

I would claim that something gets lost on that delegation.

The writing process is more than just the production of text. Many times it requires the exploration of different perspectives, thinking deeply and coming to terms that we don’t know enough about a subject and need to learn more about it.

For example, it’s essential for me to have a notebook at hand to take notes during meetings and formal discussions. I write phrases, loose words, make small diagrams, jot down some reminders. Some of them are never to be re-read again, others I revisit to structure them down into a concise structure. Most of all, they help me think about a problem.

Same holds for notes and articles. I start with a cloud of loosely related ideas, which I attempt to refine into a structured form. Similar to the double diamond process.

Same for books. Several times I’ve come to terms that I learned close to nothing about a book read one month before. Or conversations. Or movies. Or experiences.

This is, except if I reflected or acted about them. Except if I wrote down my conclusions about them.

For me, taking notes helps make sure that I’m really thinking hard about what’s in there. If I disagree with the book, sometimes it takes a long time to read the books because I’m writing so much in the margin

― Bill Gates

]]> 2023-07-02T00:00:00+01:00 Custom Pomodoro Technique https://lopespm.com/notes/2023/07/02/custom-pomodoro-technique.html https://lopespm.com/notes/2023/07/02/custom-pomodoro-technique.html

Time is the most valuable thing a man can spend.

― Theophrastus

There are countless articles, books, videos and lectures about time management. Since all of us have different needs, goals and environments, it is best to face them as guidances, where we take the ones we need, and leave the others in our toolbelt for another time.

One of these methodologies is the Pomodoro Technique, in which a kitchen timer is used to break work into intervals, typically 25 minutes in length, separated by short breaks.

This doesn’t work for me, so here is my Custom Pomodoro Technique.

How it works

I usually need long bouts of deep focus during my work or any complex tasks. This can be reading / writing code, aligning with partners, structuring plans, going through my finances, etc. 25 minutes is typically not enough, since it coincides with where most of my flow peaks happen.

One hour is my target for continuous focused work. Below that, I know that I can still take advantage of the remaining time to productively go through a problem. Above that, I start to prepare to wind down and take a break.

Rule of thumb for breaks are to be enough to feel rested, which tends to happen in less than 15 minutes.

Tools

I’ve tried several tools to track time. Physical and digital. Complex and simple. One has stuck for several years, and is the only one I use right now: Klokki

Klokki is a concise Mac application that tracks time and generates nice time reports. It also allows you to track your time automatically, but I don’t use that feature.

This is a wonderful tool that fulfills my main requirements:

  • Being able to start and stop the timer with a single click.
  • The timer should be always visible, but also non-obtrusive.
  • Simple usage, with a low UI footprint. No distractions.
  • Concise time reports, which allow me to know how much time (and energy) I’ve used throughout the week. This is important for pacing calibration.

The rules

  1. Whenever I sit down at the computer and start working, I start the timer. I converged into using a single category to track all my deep flow blocks, since having multiple categories created too much mental overhead

  2. Whenever the timer is on, I’m in productivity / work mode, and no distractions outside my goals are allowed.

  3. Refrain from stopping the flow session before the one hour mark. Start preparing to stop the flow session after the 1 hour mark.

  4. Take breaks in-between flow sessions, until I feel rested. Normally less than 15 minutes.

  5. When the timer is off, scattered tasks, exploration and wandering are allowed and encouraged.

The benefits

This is an incredibly simple methodology that I’ve been using throughout the past years. The recurrent benefits I’ve captured from it were:

  • Clear boundaries between focus time and exploration.
  • Simple, easy to follow process, allowing for flexibility on how time blocks are managed. Easy to consistently abide by.
  • Pacing: prevents over-working or under-utilization of my energy / time.

Feel free to use this guidance / methodology and experiment with it, Or not 🙏

]]> 2023-07-02T00:00:00+01:00 Solving task switching through documentation https://lopespm.com/notes/2023/05/18/solving-task-switching-through-documentation.html https://lopespm.com/notes/2023/05/18/solving-task-switching-through-documentation.html We don’t actually multitask. We switch-task, rapidly shifting from one thing to another, interrupting ourselves unproductively, and losing time in the process 1. It just so happens that many aspects of our life require managing multiple tasks at the same time, so how to approach it?

One strategy that consistently worked for me has been using documentation to checkpoint each step of a process. Just like in a videogame where the last saved checkpoint can be recalled.

The Backtracking Inefficiency

Similar to a CPU, our short-term memory (STM) has access to a limited amount (around four or seven concepts have been suggested 2) of very quick memory. Everytime we change to a different task, most of the resident concepts need to be flushed in order to load the ones from the incoming task. This is quite taxing by itself and narrowly optimizable, since our wetware is mostly fixed. The biggest efficiency opportunities lie in where these concepts are loaded from.

When working on a task, often we grab several simultaneous concepts, combine them together and generate new ones, which are then added to the concept pool and built upon further. During this process we are faced with these options:

  • Option 1. Don’t store our working memory - essentially lose work. Let’s rule out this option.
  • Option 2. Progressively store the existing concepts into our long term memory.
  • Option 3. Progressively store the existing concepts into an external memory bank, such as a document or piece of paper.

Option 2. works well, but requires significant cognitive effort and often only keeps the most relevant conclusions. If for some reason these need to be re-considered or understood why they were generated in the first place, we backtrack into the seminal thought processes, which in turn will probably require a repetition of previous work.

This is one of the biggest offenders when task switching, because even if we recall the end state of a task (which is hard by itself), a lot of time and energy are spent when revisiting the overall context via backtracking.

Documentation as Checkpoint Mechanism

Enter Option 3.: Progressively store these existing concepts into an external memory bank, such as a document or piece of paper. Although writing in an external memory bank the contents of our working memory can take some effort, the yielded results greatly justify the initial investment:

  • Less cognitive load upon re-starting a task
  • Backtracking effort is reduced, since it is a matter of reading a document
  • Shareable with other stakeholders, which shaves off the time required to explain these concepts
  • Task switching becomes smoother and less burdensome

Parting Thoughts

  • The documentation’s structure significantly impacts how efficiently it can be parsed and shared. That is a whole new topic by itself.
  • Documentation as a checkpointing mechanism is a great way to consolidate personal learnings and reflections. It’s a valuable tool for leveraging life’s experiences to mature and grow. Option 1. is such a waste.

  1. Bregman, How (and Why) to Stop Multitasking↩

  2. About the Distinction between Working Memory and Short-Term Memory↩

]]> 2023-05-18T00:00:00+01:00 The case for not having a TV https://lopespm.com/notes/2023/05/04/the-case-for-not-having-tv.html https://lopespm.com/notes/2023/05/04/the-case-for-not-having-tv.html Listen to David Sinclair or Andrew Hubberman on how to extend lifespan or improve health markers, and a common trait emerges: the body strives and improves when exposed to just enough stress and struggle. Not too much, though. Cold baths, saunas, fasting, exercise, red light (which disrupts the way that our mitochondria make energy), eating stressed greens, are examples of just that.

On the other hand, if you spend your whole day sitting, or typing, always satisfying your hunger, your AMPK and mTOR pathways (which are master regulators of cell metabolism) and sirtuins say: “Hey, times are good. Let’s just grow tissue, go forth, multiply and not build a sustainable body in the long run.”

The idolized unhurdled, stable and comfortable life, the easy access to resources and high caloric food, can do more harm than good when left unattended.

The riveting third person

I’ve consumed gargantuous amounts of television throughout my youth, so I’ve grown familiar with it’s charming alure. In an almost frictionless experience, where it suffices to lean back and pick one of the channels or series/movies/documentaries readily suggested by a streaming provider, an experience is entered where carefully crafted packages and passionate stories are delivered to us, requiring only our attention. It can even serve as the third-person to liven up the room, when in a tedious moment with a loved one.

No real struggle, no real effort required. A passive experience. The body rejoices in it, like it does when ingesting a sugar packed sweet. Just like most, I enjoy that feeling. Just like most, I have a limited amount of willpower. Hence why I don’t have a TV at home. Neither do I have sweets.

I love it. I don’t love the fact that I’m blocking myself away from those satisfactions. No, I relish the long term results. The time it opens up to actively read, write (this post for example), create, think, be present with others, and even watch podcasts (of which I make an effort to pick an episode outside the normal recommendations). I’m not perfect, and the above serves as a rule for me. And rules do have exceptions, and that’s ok.

Still, this is the case for not having a TV.

]]> 2023-05-04T00:00:00+01:00 The First Note https://lopespm.com/notes/2023/04/07/first-note.html https://lopespm.com/notes/2023/04/07/first-note.html Welcome to Notes!

This is a new space where you will be able to find essays, notes and other quick thoughts, whereas the existing Blog section will remain allocated to denser article pieces.

Why?

The Recurrent Idea

I have a rule when buying non-essential items for my house: if I’ve been recurrently thinking about buying something over a span of multiple days, then it’s a good purchasing signal. The thought process for writing articles for this blog is similar.

The downside of this approach is that the ideas and thoughts that don’t make it to a blog article are relegated into the realm of personal writings and/or conversations with others, but never exposed to the public. On the other hand, this very same thirst to share quick thoughts was recurrent by itself.

Framing

The yearning towards developing this concept was there, but a satisfactory hosting plaform was missing still:

  • Twitter: although it allowed for volume and looser thoughts, the character restriction would limit the exploration of a topic, and all the content would not be owned by me.
  • New self-hosted space: this allowed me to own the contents and have a clear separation between dense blog articles and quick thoughts, but the maintenance cost was prohibitive.

The third option was to host them here, in Byte Tank, which was customized over several years and already had the workflow and components I strived for. But how to frame this new concept into the existing structure?

The answer came from Alin Panaitiu. On his website, Alin has a separate section for Notes, which makes perfect sense to me. Notes sets the expectation that they are not fully fledged articles, and that they cater towards volume, rather than density.

Making it a reality

An idea is only worth for its implementation, and these were the last nudges that incentivized me to bring it to life:

  • The inspiring examples presented in several of the comments in Ask HN: What has your personal website/blog done for you?. One of them being a reference to Why and how to write things on the Internet, the other one being Alin’s website (see above).
  • The spirit of the earlier Internet. I remember the earlier times of the Internet where there was this widespread fascination with the fact that with a telephone connection, one could have access to its immense world, and had the possibility to create their own virtual home, that anyone could visit. That is one of the many beauties of this massive network. Low cost, virtually no gatekeepers, nearly boundless. I find this idea incredibly exciting and empowering.
  • Even in an era where text content is now produced for a low cost and in massive amounts (GPT and other LLMs), I believe there is still space for artisanal human baked opinions. The human relatableness, its connection with the environment, biology and even its limitations are still charming on their own.
  • The incentive to create notes in volume could be a great way to generate more source material that can be coalesced into new blog articles
  • Exposing looser thoughts publicly forces me to structure then, and I believe this process can make me better equipped to navigate the complex universe of human interactions, in which having a more structured mindset can make a world of a difference. Personally and professionally.

Going Forward

This is an exciting new phase for this modest establishment, and I’m looking forward to populating this section with new content. Until then!

]]> 2023-04-07T00:00:00+01:00 Book Quotes Collection - Part 1 https://lopespm.com/2022/12/29/book-quotes-collection-part-one.html https://lopespm.com/2022/12/29/book-quotes-collection-part-one.html


Books are an immensely valuable source of condensed knowledge and insightful thoughts. Here is a selection of my favorite quotes, from books I’ve read throughout the past years:



Education

  • “No man can reveal to you aught but that which already lies half asleep in the dawning of your knowledge. 13
  • By the time a student gets to college, he’s spent a decade curating a bewilderingly diverse résumé to prepare for a completely unknowable future. Come what may, he’s ready—for nothing in particular. (…) what could be a more appropriate reward for two decades of résumé-building than a seemingly elite, process-oriented career that promises to “keep options open”? 1
  • The biggest battle I have is restricting their video game time because they want to play all the time. The rule is they have to read more than they play video games. They also can’t play completely stupid video games. There’s one game they downloaded recently called Cookies or something. You literally tap a fucking cookie. It’s like a Psych 101 experiment. I made them delete the cookie game. They had to play Flappy Golf instead, which is like Flappy 2

Work

  • But I say to you that when you work you fulfil a part of earth’s furthest dream, assigned to you when that dream was born, And in keeping yourself with labour you are in truth loving life, And to love life through labour is to be intimate with life’s inmost secret. 13
  • Promises are easy and cheap to make, actual work is hard and expensive. If it wasn’t, you’d just have done it now rather than promised it later. 11

Business / Companies

  • Startups Are Easy, Stayups Are Hard 11
  • On describes the earliest startup as like driving a race car. You’re close to the ground, and you feel every move you make. You have control, you can turn quickly, you feel like things are moving fast. Of course, you’re also at risk of crashing at any moment, but you only take yourself down if you do. As you grow, you graduate to a commercial flight. You’re farther from the ground, and more people’s lives depend on you, so you need to consider your movements more carefully, but you still feel in control and can turn the plane relatively quickly. Finally, you graduate to a spaceship, where you can’t make quick moves and the course is set long in advance, but you’re capable of going very far and taking tons of people along for the ride. 16
  • Most of the people at PayPal don’t understand this. The reason it worked was because the cost of transactions in PayPal was lower than any other system. And the reason the cost of transactions was lower is because we were able to do an increasing percentage of our transactions as ACH, or automated clearinghouse, electronic transactions, and most importantly, internal transactions. Internal transactions were essentially fraud-free and cost us nothing. An ACH transaction costs, I don’t know, like twenty cents or something. But it was slow, so that was the bad thing. It’s dependent on the bank’s batch processing time. And then the credit card transaction was fast, but expensive in terms of the credit card processing fees and very prone to fraud. That’s the problem Square is having now. “Square is doing the wrong version of PayPal. The critical thing is to achieve internal transactions. 2
  • Is this thing any good? Does it solve a real problem? Should we have made it better? Are we making what customers want? Is anybody going to buy this? Did we price it right? 11
  • There’s only a bunch of business-axiom baloney like “If you’re not growing, you’re dying.” Says who? 11
  • Tolstoy opens Anna Karenina by observing: “All happy families are alike; each unhappy family is unhappy in its own way.” Business is the opposite. All happy companies are different: each one earns a monopoly by solving a unique problem. All failed companies are the same: they failed to escape competition. 1
  • Are production incidents happening that are taking up lots of time? Are a bunch of people sick? Are they bickering over coding style in their code review comments? Are the tickets that are being written vague, too big, too small? Does the team seem upbeat in their communication style, sharing fun things as well as important work in chat, or are they purely business? Look at their calendars. Is the team spending many hours a week in meetings? Is their manager not doing 1-1s? None of these things are necessarily smoking guns, but they may point to an area to address. 16
  • Conway’s Law is often cited in discussions of this kind of structure. It states: “Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” 16
  • As ridiculous and dangerous as this might seem, plenty of teams fall prey to the lure of status. These often include altruistic nonprofit organizations that come to believe that the nobility of their mission is enough to justify their satisfaction. Political groups, academic departments, and prestigious companies are also susceptible to this dysfunction, as they often see success in merely being associated with their special organizations. 17

Planning

  • Forget “minimum viable products”—ever since he started Apple in 1976, Jobs saw that you can change the world through careful planning, not by listening to focus group feedback or copying others’ successes. 1
  • Because let’s face it: Goals are fake. Nearly all of them are artificial targets set for the sake of setting targets. 11
  • Start with a shared understanding of the goals, risks, and the questions to answer before making a decision. When you assign the ownership for making a 16
  • “We who cut mere stones must always be envisioning cathedrals.” 9
  • You have 10 productive engineering weeks per engineer per quarter 16
  • Dedicate 20% of your team’s schedule to “sustaining engineering.” 16
  • It’s likely that Q1 (immediately after the winter holidays) will be the most productive and Q4 (the quarter that includes winter and the end-of-year holidays) will be the least productive. 16
  • Therefore, you must always be aggressive about sharing estimates and updates to estimates, even when people don’t ask, especially if you believe that the project is critical or likely to take longer than a few weeks. This means you must be aggressive about getting estimates, and as we all know, software estimation is a very difficult process. Negotiating the process that your team uses to estimate, on what timescale, for what projects, may be part of your job at this level. 16
  • If you haven’t taken the time to articulate the value of this work, it will get pushed aside in favor of projects that are more clearly valuable. 16
  • My advice is to do your best to gather data to support yourself, and talk about what will be possible when the work is done. 16
  • When an engineer comes to you with an engineering project that she wants to do, think about framing the project by answering these questions: How big is that project? How important is it? Can you articulate the value of that project to anyone who asks? What would successful completion of the project mean for the team? 16
  • The hardest part, in many ways, was getting started. The second-hardest part was getting comfortable making a guess about the future with highly imperfect information. 16
  • “The key, of course, is to define our goals, our results, in a way that is simple enough to grasp easily, and specific enough to be actionable. Profit is not actionable enough. It needs to be more closely related to what we do on a daily basis. (…) By combining some and eliminating others, they narrowed them to seven: revenue, expenses, new customer acquisition, current customer satisfaction, employee retention, market awareness, and product quality. 17

Management

  • The most important lesson I’ve learned is that you have to be able to manage yourself if you want to be good at managing others. The more time you spend understanding yourself, the way you react, the things that inspire you, and the things that drive you crazy, the better off you will be. 16
  • Contrary to the notion that teams waste time and energy arguing, those that avoid conflict actually doom themselves to revisiting issues again and again without resolution. They often ask team members to take their issues “off-line,” which seems to be a euphemism for avoiding dealing with an important topic, only to have it raised again at the next meeting. 17
  • Great teams ensure that everyone’s ideas are genuinely considered, which then creates a willingness to rally around whatever decision is ultimately made by the group. And when that is not possible due to an impasse, the leader of the team is allowed to make the call. 17
  • (…) dysfunctional teams that try to hedge their bets and delay important decisions until they have enough data to feel certain that they are making the right decision. As prudent as this might seem, it is dangerous because of the paralysis and lack of confidence it breeds within a team. 17
  • More than any other member of the team, the leader must be comfortable with the prospect of making a decision that ultimately turns out to be wrong. And the leader must be constantly pushing the group for closure around issues, as well as adherence to schedules that the team has set. What the leader cannot do is place too high a premium on certainty or consensus. 17
  • (…) the most effective and efficient means of maintaining high standards of performance on a team is peer pressure. One of the benefits is the reduction of the need for excessive bureaucracy around performance management and corrective action. More than any policy or system, there is nothing like the fear of letting down respected teammates that motivates people to improve their performance. 17
  • If the boss really wants to know what’s going on, the answer is embarrassingly obvious: They have to ask! Not vague, self-congratulatory bullshit questions like “What can we do even better?” but the hard ones like “What’s something nobody dares to talk about?” or “Are you afraid of anything at work?” or “Is there anything you worked on recently that you wish you could do over?” Or even more specific ones like “What do you think we could have done differently to help Jane succeed?” or “What advice would you give before we start on the big website redesign project?” 11
  • A lack of engagement in meetings tends to mean the team isn’t engaged by the work or do not feel like they have a say in the decision-making process. 16
  • But there is such a thing as artificial harmony, and conflict-avoidant managers tend to favor harmony above functional working relationships. 16
  • The goal is to identify problems that are causing the team to work less effectively together and resolve them, not to become the team’s therapist. 16
  • Many of us believe that the way to be liked is to be seen as nice — that niceness is itself the goal. Your goal as a manager, however, should not be to be nice, it should be to be kind. 16
  • It’s kind to tell someone who isn’t ready for promotion that she isn’t ready, and back that up with the work she needs to do to get there. It’s unkind to lead that person on, saying “Maybe you could get promoted,” and then watch her fail. It’s kind to tell someone that his behavior in meetings is disrupting the group. It’s awkward, and uncomfortable, but it’s also part of your job as his manager to have these difficult conversations. 16
  • Thinking about your actions is the best way to combat fear of conflict. Am I pushing this decision to the team because they really are the best people to decide, or am I just afraid that if I make an unpopular but necessary decision people will be mad at me? Am I avoiding working through this issue with my peer because she’s truly difficult to work with, or am I just hoping that the issue will resolve itself because I don’t want to have to discuss it and possibly be wrong? Am I holding back on giving my employee this feedback because he really was having a bad day and it’s just a one-off, or am I holding back because I’m afraid he won’t like me as a manager if I tell him? Be thoughtful about your behavior, and it’s unlikely that you’ll seek out unnecessary conflict. 16
  • The real goal here is psychological safety — that is, a team whose members are willing to take risks and make mistakes in front of one another. 16
  • Your first goal is to protect your team as a whole, the second is to protect each individual on the team, and your last priority is protecting yourself. 16
  • Simply put, if your team member doesn’t respect you or her peers, why is she working there? 16
  • So at the end of another day when I feel like I didn’t write enough code and I have no way to quantify what I’ve achieved, I tell myself I was being as good a manager as I know how to be. 16
  • Project management. Onboarding new team members. Working with the product team to break down product roadmap goals into technical deliverables. Production support. These are all skills members of your team need to learn. 16
  • Even when you have been hired to fix a team, remember that the company has gotten this far because of some fundamental strengths. 16
  • Whether it’s QA, design, product management, or technical operations, ask lots of questions, but in an open way. Make it clear to the person that your goal is to understand what she does so that you’re capable of appreciating it better. 16
  • Sit in their meetings. Are they boring to you? Is the team bored? Who is speaking most of the time? Are there regular meetings with the whole team where the vast majority of the time is spent listening to the manager or product lead talk? Boring meetings are a sign. They may be a sign of inefficient planning on the part of the organizers. There may be too many meetings happening for the information covered. They may indicate that the team members don’t feel they can actually help set the direction of the team, or choose the work that will happen. They often signal a lack of healthy conflict on a team. Good meetings have a heavy discussion element, where opinions and ideas are drawn out of the team. If the meetings are overscripted, so that no real conversation can take place, it stifles that creative discussion. If people are afraid to disagree or bring up issues for fear of dealing with conflict, or if managers always shut down conflict without letting disagreements air, this is a sign of an unhealthy team culture. 16
  • A bunch of people who never talk to each other and are always working on independent projects are not really working as a team. There would be nothing wrong with that if the team were performing well, but given that they’re not, this may be contributing to your problem. 16
  • Projects change. Teams may even be disbanded or moved around in ways that you don’t understand or agree with. As a manager, the best thing you can do is help people feel capable of tying up loose ends, stabilizing the current in-flight projects, and easing into their new work in a controlled fashion. This is an area where you can and should push back. Make sure that your teams get adequate time to finish up current work. Furthermore, push for engineering involvement in the early planning for the new work so that people can get excited about the projects they are moving on to. 16
  • It’s not about being the lead engineer, chasing the latest language or framework, or having the shiniest technology. It’s about building a team with the tools and attributes to build the best possible product for our customers. 16
  • You can’t give up the responsibility of management without giving up the power that comes with it. 16
  • Reach through the part of you that is shy about praising people or embarrassed to share your feelings, and go into the part of you that cares about the people you work with. You can share these stories in a way that is not forced or fake. 16
  • (…) the larger the company, the smaller the team should be at the top. And with the addition of a new head of sales and a human resources director, her staff had grown to a barely manageable eight. It wasn’t that Kathryn couldn’t handle the weekly one-on-ones, but it was increasingly difficult to have fluid and substantive discussions during staff meetings with nine people sitting around the table. 17
  • (…) trust is the confidence among team members that their peers’ intentions are good, and that there is no reason to be protective or careful around the group. (…) Teams that lack trust waste inordinate amounts of time and energy managing their behaviors and interactions within the group. They tend to dread team meetings, and are reluctant to take risks in asking for or offering assistance to others. As a result, morale on distrusting teams is usually quite low, and unwanted turnover is high. 17

Culture

  • Initially the technologists and the hippies did not interface well. Many in the counterculture saw computers as ominous and Orwellian, the province of the Pentagon and the power structure. In The Myth of the Machine, the historian Lewis Mumford warned that computers were sucking away our freedom and destroying “life-enhancing values.” An injunction on punch cards of the period—“Do not fold, spindle or mutilate”—became an ironic phrase of the antiwar Left. 3
  • Engineers frequently struggle with the transition to respecting and communicating well with diverse peers. I believe the struggle with respect is a side effect of the current tech culture, which tells us that engineers are the smartest people in the room. It can’t be said strongly enough: your peers who are not analytically driven are not stupid. 16
  • “Engineers who graduated from MIT” is not a culture. “People who value technology innovation, hard work, intellect, scientific process, and data” might be. 16
  • A lot of interviews try to determine cultural fit by what I would call “friendship” markers, such as “Would you like being stuck in an airport with this person?” 16
  • Furthermore, culture fit as determined by friendship tests is almost certain to be discriminatory in some way. Humans form friendships with people who have significant shared background experiences, and these experiences tend to closely correlate with things like schooling, race, class, and gender. The shortcuts you get by hiring friends are not usually the values you need to form a strong team. 16

Systems / Processes

  • A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. 16
  • As your teams and systems grow, it’s almost impossible for any one person to keep the systems in her head. Because we have a bunch of people coordinating work, we evolve processes around that work coordination in order to make risks obvious. 16
  • (…) you should not put a complicated process on any activity where you want people to move quickly and where you believe the risk for change in that activity is low or that the risks themselves are obvious to the whole team. If you want to do code review for all changes, make sure that the process for code review is not so onerous that the team slows down significantly on minor changes, because that will impact your whole group’s productivity. The second implication is that you need to be on the lookout for places where there is hidden risk, and draw those hidden risks out into the open. There’s a saying in politics that “a good political idea is one that works well in half-baked form,” and the same goes for engineering processes. The processes should have value even when they are not followed perfectly, and that value should largely lie in the act of socializing change or risk to the team as a whole. 16
  • Be specific about the kinds of changes that need architecture review. Usually these include new languages, new frameworks, new storage systems, and new developer tooling. 16

Focus on what is important

  • But in effect it was unanswerable even now, since the few scattered survivors from the ancient world were incapable of comparing one age with another. They remembered a million useless things, a quarrel with a workmate, a hunt for a lost bicycle pump, the expression on a long-dead sister’s face, the swirls of dust on a windy morning seventy years ago: but all the relevant facts were outside the range of their vision. They were like the ant, which can see small objects but not large ones. 7
  • It’s lonely at the top. Ninety-nine percent of people in the world are convinced they are incapable of achieving great things, so they aim for the mediocre. The level of competition is thus fiercest for “realistic” goals, paradoxically making them the most time and energy-consuming. It is easier to raise \$1,000,000 than it is \$100,000. It is easier to pick up the one perfect 10 in the bar than the five 8s. If you are insecure, guess what? The rest of the world is, too. Do not overestimate the competition and underestimate yourself. You are better than you think. 8
  • One does not accumulate but eliminate. It is not daily increase but daily decrease. The height of cultivation always runs to simplicity. 8
  • “Five years of reporting on attention have confirmed some home truths,” Gallagher reports. “[Among them is the notion that] ‘the idle mind is the devil’s workshop’ when you lose focus, your mind tends to fix on what could be wrong with your life instead of what’s right.” 9
  • You have a finite amount of willpower that becomes depleted as you use it. 9
  • Urgency is often more clearly felt than importance. 16
  • “If everything is important, then nothing is.” 17

Choices / Decisions

  • “Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to,” said the Cat. “I don’t much care where —” said Alice. “Then it doesn’t matter which way you go,” said the Cat (quoted from Lewis Carroll, Alice in Wonderland) 8
  • (…) a decision is better than no decision. They also realize that it is better to make a decision boldly and be wrong—and then change direction with equal boldness—than it is to waffle. 17
  • If you’re 25 and announce you want to start a new circus, the response is different: Be realistic; become a lawyer or an accountant or a doctor, have babies, and raise them to repeat the cycle. 8
  • It was curious how that predestined horror moved in and out of one’s consciousness. There it lay, fixed in future times, preceding death as surely as 99 precedes 100. One could not avoid it, but one could perhaps postpone it: and yet instead, every now and again, by a conscious, wilful act, one chose to shorten the interval before it happened. 7
  • Much of your pain is self-chosen. 13
  • No is easier to do, yes is easier to say. No is no to one thing. Yes is no to a thousand things. (…) No is calm but hard. Yes is easy but a flurry. Knowing what you’ll say no to is better than knowing what you’ll say yes to. 11
  • (…) so practice getting comfortable with the quick no (and the quick yes!) for low-risk, low-impact decisions. 16
  • When you want to allow for group decision making, the group needs to have a clear set of standards that they use to evaluate decisions. Start with a shared understanding of the goals, risks, and the questions to answer before making a decision. When you assign the ownership for making a decision to someone on the team, make it clear which members of the team should be consulted for feedback and who needs to be informed of the decision or plan. 16
  • Taking conflicting perspectives and incomplete information and setting a direction, knowing that the consequences of a poor decision will impact both you and possibly the whole team. If making decisions were easy, there would be much less need for managers and leaders. However, as anyone who has spent a lot of time managing can tell you, making decisions is one of the most draining and stressful parts of the job. 16

Get Stuff Done / Productivity

  • People are fond of using the “it’s not what you know, it’s who you know” adage as an excuse for inaction, as if all successful people are born with powerful friends. 8
  • Tomorrow becomes never. No matter how small the task, take the first step now! 8
  • It’s not uncommon for people to pick up their phones dozens of times a day when some push notification makes it buzz, because WHAT IF IT WAS SOMETHING SUPER IMPORTANT! (It just about never is.) 11
  • “There is nothing so useless as doing efficiently that which should not be done at all.” 11
  • “I know some of my classmates pursued the alternative scattershot technique with some success, but that’s not my bag,” Marrinan said, explaining his perseverance. “I deal with rejection by persisting, not by taking my business elsewhere. My maxim comes from Samuel Beckett, a personal hero of mine: ‘Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.’ You won’t believe what you can accomplish by attempting the impossible with the courage to repeatedly fail better.” 8
  • Success is not a matter of mastering subtle, sophisticated theory, but rather of embracing common sense with uncommon levels of discipline and persistence. 17

Solving problems

  • She provides the example of a couple fighting over inequitable splitting of household chores. “Rather than continuing to focus on your partner’s selfishness and sloth,” she suggests, “you might focus on the fact that at least a festering conflict has been aired, which is the first step toward a solution to the problem, and to your improved mood.” 9
  • Your boss may not want to be stuck solving your problems, but you can bet that she’ll be happy to provide feedback if you phrase it as needing advice. 16

Accountabilty

  • “I’m using you as an example, Carlos, because you are an easy person to let off the hook. But this could apply to anyone. Some people are hard to hold accountable because they are so helpful. Others because they get defensive. Others because they are intimidating. I don’t think it’s easy to hold anyone accountable, not even your own kids.” 17
  • “You are fighting. But about issues. That’s your job. Otherwise, you leave it to your people to try to solve problems that they can’t solve. They want you to hash this stuff out so they can get clear direction from us.” 17
  • In fact, team members who are particularly close to one another sometimes hesitate to hold one another accountable precisely because they fear jeopardizing a valuable personal relationship. Ironically, this only causes the relationship to deteriorate as team members begin to resent one another for not living up to expectations and for allowing the standards of the group to erode. 17

Communication

  • In my experience, most people need to hear something at least three times before it really sinks in. You’re going to tell your own senior management and leadership team. You’ll hold an all hands meeting. You may need to send some email detailing the changes as well. A little bit of communication planning can go a long way in such situations. Try to anticipate the questions you might get and prepare answers for those questions. Be as clear as possible about the projects or structure to be changed, so there’s little room for confusion. And don’t forget to sell this change as a good thing! 16

Human condition

  • ‘When you make love you’re using up energy; and afterwards you feel happy and don’t give a damn for anything. They can’t bear you to feel like that. They want you to be bursting with energy all the time. All this marching up and down and cheering and waving flags is simply sex gone sour. If you’re happy inside yourself, why should you get excited about Big Brother and the Three-Year Plans and the Two Minutes Hate and all the rest of their bloody rot?’ 7
  • Having so recently been one of the underdogs of the savannah, we are full of fears and anxieties over our position, which makes us doubly cruel and dangerous. 5
  • I define masculinity as that part of a man that equips him to survive as an individual, clan, and species. Without this masculine energy we would have all become extinct eons ago. Masculinity empowers a man to create and produce. It also empowers him provide for and protect those who are important to him. These aspects of masculinity include strength, discipline, courage, passion, persistence, and integrity. 10
  • This frustration is due to the reality that in general, women view men who try to please them as weak and hold these men in contempt. Most women do not want a man who tries to please them — they want a man who knows how to please himself. Women consistently share with me that they don’t want a passive, pleasing wimp. They want a man — someone with his balls still intact 10
  • When recovering Nice Guys create relationships with people who don’t need fixing, they improve their odds of finding the love they want. This doesn’t mean searching for the perfect partner, just one who is already taking responsibility for her own life. 10
  • Internalized shame and fear are the greatest barriers to a satisfying sex life. A man can read all the books he wants on “How To Pick Up Women” or watch all the instructional videos on improving sexual technique. None of these things will help him get good sex as long as he has shame and fear about being sexual or being a sexual being. 10
  • “I am pretty afraid to fail. And so I tend to over-engineer things and do them myself. I don’t like to tell other people what to do, which, ironically, only makes it more likely that I’m going to fail.” For the slightest moment, Jeff seemed to fight back emotions and then recovered instantly. He was sure no one noticed. “And I think that’s probably the biggest reason that we haven’t succeeded, and that I’m not the CEO anymore.” He paused, and then added quickly, “Which I’m okay with, really. In fact, I’m pretty happy to be out of that job.” 17

Relationships

  • All great relationships, the ones that last over time, require productive conflict in order to grow. This is true in marriage, parenthood, friendship, and certainly business. 17
  • And when one of you falls down he falls for those behind him, a caution against the stumbling stone. Ay, and he falls for those ahead of him, who though faster and surer of foot, yet removed not the stumbling stone. 13
  • Your friend is your needs answered. (…) For that which you love most in him may be clearer in his absence, as the mountain to the climber is clearer from the plain. And let there be no purpose in friendship save the deepening of the spirit. (…) For it is his to fill your need, but not your emptiness. 13
  • Former couples who break up because they don’t see a future together can remain friends, and the same goes for former employees who simply need a different team or company to shine. 16
  • “Two households, both alike in dignity.” The two houses are alike, yet they hate each other. They grow even more similar as the feud escalates. Eventually, they lose sight of why they started fighting in the first place. 1
  • All the evidence in the world can’t change a person who doesn’t want to change. 16
  • A very common clash occurs between people who are extremely analytically driven and those who are more creatively or intuitively focused. Another is between the people who prefer to embrace agility and change (and, yes, sometimes disorder) and those who push for more long-term planning, deadlines, and budgets. 16
  • “Set me like a seal upon thy heart, love is as strong as death.” 12
  • For even as love crowns you so shall he crucify you. Even as he is for your growth so is he for your pruning. 13
  • But if in your fear you would seek only love’s peace and love’s pleasure, Then it is better for you that you cover your nakedness and pass out of love’s threshing-floor, Into the seasonless world where you shall laugh, but not all of your laughter, and weep, but not all of your tears. 13
  • Your children are not your children. (…) You are the bows from which your children as living arrows are sent forth. 13

Behaviour

  • She knew the whole drivelling song by heart, it seemed. Her voice floated upward with the sweet summer air, very tuneful, charged with a sort of happy melancholy. One had the feeling that she would have been perfectly content, if the June evening had been endless and the supply of clothes inexhaustible, to remain there for a thousand years, pegging out diapers and singing rubbish. It struck him as a curious fact that he had never heard a member of the Party singing alone and spontaneously. It would even have seemed slightly unorthodox, a dangerous eccentricity, like talking to oneself. Perhaps it was only when people were somewhere near the starvation level that they had anything to sing about. 7
  • As recovering Nice Guys begin to surrender, dwell in reality, express their feelings, face their fears, develop integrity, and set boundaries, they access a power that allows them to welcome and embrace the challenges and “gifts” of life. Life isn’t a merry-go-round, it’s a roller coaster. As they reclaim personal power, recovering Nice Guys can experience the world in all of its serendipitous beauty. Life won’t always be smooth, it may not always be pretty, but it will be an adventure — one not to be missed. 10
  • As Nice Guys try to avoid the dark side of their masculinity, they also repress many other aspects of this male energy force. As a result, they often lose their sexual assertiveness, competitiveness, creativity, ego, thirst for experience, boisterousness, exhibitionism, and power. 10
  • (Activities to break free from the Nice Guy Syndrome) 10
    • If it frightens you, do it.
    • Don’t settle. Every time you settle, you get exactly what you settled for.
    • Put yourself first.
    • No matter what happens, you will handle it.
    • Whatever you do, do it 100%.
    • If you do what you have always done, you will get what you have always got.
    • You are the only person on this planet responsible for your needs, wants, and happiness.
    • Ask for what you want.
    • If what you are doing isn’t working, try something different.
    • Be clear and direct.
    • Learn to say “no.”
    • Don’t make excuses.
    • If you are an adult, you are old enough to make your own rules.
    • Let people help you.
    • Be honest with yourself.
    • Do not let anyone treat you badly. No one. Ever.
    • Remove yourself from a bad situation instead of waiting for the situation to change.
    • Don’t tolerate the intolerable — ever.
    • Stop blaming. Victims never succeed.
    • Live with integrity. Decide what feels right to you, then do it.
    • Accept the consequences of your actions.
    • Be good to yourself.
    • Think “abundance.”
    • Face difficult situations and conflict head on.
    • Don’t do anything in secret.
    • Do it now.
    • Be willing to let go of what you have so you can get what you want.
    • Have fun. If you are not having fun, something is wrong.
    • Give yourself room to fail. There are no mistakes, only learning experiences.
    • Control is an illusion. Let go; let life happen.
  • For to be idle is to become a stranger unto the seasons, and to step out of life’s procession, that marches in majesty and proud submission towards the infinite. 13
  • Have you peace, the quiet urge that reveals your power? Have you remembrances, the glimmering arches that span the summits of the mind? Have you beauty, that leads the heart from things fashioned of wood and stone to the holy mountain? Tell me, have you these in your houses? Or have you only comfort, and the lust for comfort, that stealthy thing that enters the house a guest, and then becomes a host, and then a master? 13
  • (about crime and punishment) And when the black thread breaks, the weaver shall look into the whole cloth, and he shall examine the loom also. 13
  • If either your sails or your rudder be broken, you can but toss and drift, or else be held at a standstill in mid-seas. For reason, ruling alone, is a force confining; and passion, unattended, is a flame that burns to its own destruction. (…) And since you are a breath in God’s sphere, and a leaf in God’s forest, you too should rest in reason and move in passion. ​​13
  • And the longer it lasted the more deadly it became. “It is as if I had been going downhill while I imagined I was going up. And that is really what it was. I was going up in public opinion, but to the same extent life was ebbing away from me. And now it is all done and there is only death. 14
  • Above all, don’t lie to yourself. The man who lies to himself and listens to his own lie comes to such a pass that he cannot distinguish the truth within him, or around him, and so loses all respect for himself and for others. 15
  • There is only one means of salvation, then take yourself and make yourself responsible for all men’s sins, that is the truth, you know, friends, for as soon as you sincerely make yourself responsible for everything and for all men, you will see at once that it is really so, and that you are to blame for every one and for all things. But throwing your own indolence and impotence on others you will end by sharing the pride of Satan and murmuring against God. 15
  • And people have even ceased to feel the impulse to self‐criticism. Don’t be like everyone else, even if you are the only one.” 15
  • Don’t be afraid. Conflict avoidance often arises from fear. We’re scared of the responsibility of making the decision. We’re afraid of seeming too demanding. We’re afraid people will quit if we give them uncomfortable feedback. We’re afraid people won’t like us, or that we’ll fail when we take this risk. Some fear is natural, and being sensitive to the outcomes of conflict is a wise habit. 16
  • Conflict avoidance runs deep in most of us, and matters of what feels like personal style can be particularly difficult. 16
  • “I don’t think anyone ever gets completely used to conflict. If it’s not a little uncomfortable, then it’s not real. The key is to keep doing it anyway.” 17
  • This background, combined with a strong desire to be seen as smart and “right,” sometimes led to some less-than-productive conversations about technology directions. I’d find myself arguing the merits of a language choice or technology purely based on the technical merits. When this happened I became another engineer arguing in a group of engineers. It took me a long time to realize that my job wasn’t to be the smartest person in the room. It wasn’t to be “right.” Rather, my role was to help the team make the best possible decisions and help them implement them in a sustainable and efficient way. 16
  • “Wanting to be a CTO (or VP of Engineering) is like wanting to be married. Remember that it’s not just the title, it’s also the company and the people that matter.” Titles are definitely not everything. 16
  • Throwing out jargon to people who aren’t familiar with it — and who don’t even need to be familiar with it — makes us look stupid to them. 16
  • “Okay, first of all, don’t ever slam one of your teammates when that person isn’t in the room. I don’t care what you think of Mikey. She is part of this team, and you have to take your issues to her directly, or to me. You’re going to have to make that right.” 17

Group Thinking / Living in Society

  • They were talking about the Lottery. Winston looked back when he had gone thirty metres. They were still arguing, with vivid, passionate faces. The Lottery, with its weekly pay-out of enormous prizes, was the one public event to which the proles paid serious attention. It was probable that there were some millions of proles for whom the Lottery was the principal if not the only reason for remaining alive. It was their delight, their folly, their anodyne, their intellectual stimulant. Where the Lottery was concerned, even people who could barely read and write seemed capable of intricate calculations and staggering feats of memory. There was a whole tribe of men who made a living simply by selling systems, forecasts, and lucky amulets. 7
  • Whenever you find yourself on the side of the majority, it is time to pause and reflect. 8
  • It takes a tribe to raise a human. 5
  • Sociological research has shown that the maximum ‘natural’ size of a group bonded by gossip is about 150 individuals. 5
  • (…) for the craving for universal unity is the third and last anguish of men. Mankind as a whole has always striven to organize a universal state. 15
  • “What do you mean by isolation?” I asked him. “Why, the isolation that prevails everywhere, above all in our age—it has not fully developed, it has not reached its limit yet. For every one strives to keep his individuality as apart as possible, wishes to secure the greatest possible fullness of life for himself; but meantime all his efforts result not in attaining fullness of life but self‐destruction, for instead of self‐realization he ends by arriving at complete solitude. All mankind in our age have split up into units, they all keep apart, each in his own groove; each one holds aloof, hides himself and hides what he has, from the rest, and he ends by being repelled by others and repelling them. He heaps up riches by himself and thinks, ‘How strong I am now and how secure,’ and in his madness he does not understand that the more he heaps up, the more he sinks into self‐destructive impotence. For he is accustomed to rely upon himself alone and to cut himself off from the whole; he has trained himself not to believe in the help of others, in men and in humanity, and only trembles for fear he should lose his money and the privileges that he has won for himself. Everywhere in these days men have, in their mockery, ceased to understand that the true security is to be found in social solidarity rather than in isolated individual effort. 15
  • “Madness is rare in individuals—but in groups, parties, nations, and ages it is the rule,” Nietzsche wrote (before he went mad). If you can identify a delusional popular belief, you can find what lies hidden behind it: the contrarian truth. 1

Thoughts / Mindset

  • “Shallow men believe in luck, believe in circumstances. Strong men believe in cause and effect.” 1
  • (…) none of these things exists outside the stories that people invent and tell one another. There are no gods in the universe, no nations, no money, no human rights, no laws, and no justice outside the common imagination of human beings. 5
  • “He who is abandoned is an abandoner,” 3
  • The invitation read, “There’s an old Hindu saying that goes, ‘In the first 30 years of your life, you make your habits. For the last 30 years of your life, your habits make you.’ Come help me celebrate mine.” 3
  • (…) and more importantly, in order to take full advantage of the how, you need to understand the why. 4
  • Will they reach the nursery in time? If so, how delightful for them, and we shall all breathe a sigh of relief, but there will be no story. On the other hand, if they are not in time, I solemnly promise that it will all come right in the end. 6
  • What are you putting off out of fear? Usually, what we most fear doing is what we most need to do. 8
  • The existential vacuum manifests itself mainly in a state of boredom. 8
  • There is a direct correlation between an increased sphere of comfort and getting what you want. 8
  • I define personal power as a state of mind in which a person is confident he can handle whatever may come. This kind of power not only successfully deals with problems, challenges and adversity, it actually welcomes them, meets them head on, and is thankful for them. Personal power isn’t the absence of fear. Even the most powerful people have fear. Personal power is the result of feeling fear, but not giving in to the fear. 10
  • “Comparison is the death of joy.” 11
  • Yet the timeless in you is aware of life’s timelessness, And knows that yesterday is but today’s memory and tomorrow is today’s dream. 13
  • Men will even give their lives if only the ordeal does not last long but is soon over, with all looking on and applauding as though on the stage. 15
  • (…) for all is like an ocean, all is flowing and blending; a touch in one place sets up movement at the other end of the earth. 15
  • (…) fools are made for wise men’s profit.” 15
  • “There’s no such thing as a well-adjusted public figure. If they were well adjusted they wouldn’t try to be a public figure.” 2
  • “Men love the downfall and disgrace of the righteous” 15
  • Goodness is one thing with me and another with a Chinaman, so it’s a relative thing. Or isn’t it? Is it not relative? A treacherous question! 15
  • (…) “happiness and positivity is a choice,” the teammate who pooh-poohs every idea and criticizes everything is going to have problems fitting in. Sometimes, people will change to adopt the values. 16
  • “Politics is when people choose their words and actions based on how they want others to react rather than based on what they really think.” 17
  • Ideological conflict is limited to concepts and ideas, and avoids personality-focused, mean-spirited attacks. 17
  • You talk when you cease to be at peace with your thoughts; And when you can no longer dwell in the solitude of your heart you live in your lips, and sound is a diversion and a pastime. And in much of your talking, thinking is half murdered. 13

Wealth

  • And what follows from this right of multiplication of desires? In the rich, isolation and spiritual suicide; in the poor, envy and murder; for they have been given rights, but have not been shown the means of satisfying their wants. They maintain that the world is getting more and more united, more and more bound together in brotherly community, as it overcomes distance and sets thoughts flying through the air. 15
  • (…) yet to the very end he persisted in hoping that he would get that three thousand, that the money would somehow come to him of itself, as though it might drop from heaven. That is just how it is with people who, like Dmitri, have never had anything to do with money, except to squander what has come to them by inheritance without any effort of their own, and have no notion how money is obtained. 15

History / Evolution

  • Just 6 million years ago, a single female ape had two daughters. One became the ancestor of all chimpanzees, the other is our own grandmother. 5
  • Women who gave birth earlier, when the infants brain and head were still relatively small and supple, fared better and lived to have more children. Natural selection consequently favoured earlier births. And, indeed, compared to other animals, humans are born prematurely, when many of their vital systems are still under-developed. A colt can trot shortly after birth; a kitten leaves its mother to forage on its own when it is just a few weeks old. Human babies are helpless, dependent for many years on their elders for sustenance, protection and education. 5
  • Some scholars believe there is a direct link between the advent of cooking, the shortening of the human intestinal track, and the growth of the human brain. 5
  • It is unsettling – and perhaps thrilling – to think that we Sapiens could at one time have sex with an animal from a different species, and produce children together. 5

Religion / Spirituality

  • “I think different religions are different doors to the same house. Sometimes I think the house exists, and sometimes I don’t. It’s the great mystery.” 3
  • For even those who have renounced Christianity and attack it, in their inmost being still follow the Christian ideal 15
  • Freedom, free thought and science, will lead them into such straits and will bring them face to face with such marvels and insoluble mysteries, that some of them, the fierce and rebellious, will destroy themselves, others, rebellious but weak, will destroy one another, while the rest, weak and unhappy, will crawl fawning to our feet and whine to us: “Yes, you were right, you alone possess His mystery, and we come back to you, save us from ourselves!” 15

Geopolitics

  • “Every morning at six A.M., the first thing to hit my desk was this overnight report that included information on who got killed and what killed them,” O’Connell said. “I kept thinking, This is insane. Why are we in this place? It was not just Iraq but the whole picture. Why were we so invested in that part of the world?” The unsurprising answer that O’Connell came up with was oil. 2

Future

  • In Cowen’s future, huge gains in artificial intelligence will lead to the elimination of many of today’s high-employment lines of work. The people who thrive in this environment will be very bright and able to complement the machines and team effectively with them. As for the unemployed masses? Well, many of them will eventually find jobs going to work for the Haves, who will employ teams of nannies, housekeepers, and gardeners. If anything Musk is doing might alter the course of mankind toward a rosier future, Cowen can’t find it. Coming up with true breakthrough ideas is much harder today than in the past, according to Cowen, because we’ve already mined the bulk of the big discoveries. 2
  • If not, things will not improve by themselves. If you are kidding yourself, it is time to stop and plan for a jump. Barring any James Dean ending, your life is going to be LONG. Nine to five for your working lifetime of 40–50 years is a long-ass time if the rescue doesn’t come. About 500 months of solid work. How many do you have to go? It’s probably time to cut your losses. 8

Off the Cuff

  • (Musk nearly died on a trip to Africa, where he caught a type of severe malaria. It was discovered late on his return to the United States, and Musk spent 10 agonizing days in intensive care. It took him six months to recover and lost 20 kilos during the illness.) “I came very close to dying,” Musk said. “That’s my lesson for taking a vacation: vacations will kill you.” 2
  • “The guy comes in, and Elon asks him why they’re meeting,” Spikes said. “He said, ‘To develop a relationship.’ Elon replied, ‘Okay. Nice to meet you,’ which basically meant, ‘Get the fuck out of my office.’ 2



Note that the following are affiliate links, and as an Amazon Associate I earn from qualifying purchases whose commissions help this small establishment, at no additional cost to you:

  1. Zero to One: Notes on Start Ups, or How to Build the Future (Masters, Blake;Thiel, Peter) ↩

  2. Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future (Ashlee Vance) ↩

  3. Steve Jobs (Isaacson, Walter) ↩

  4. Methods of Persuasion: How to Use Psychology to Influence Human Behavior (Kolenda, Nick) ↩

  5. Sapiens: A Brief History of Humankind (Yuval Noah Harari) ↩

  6. Peter Pan (J. M. Barrie) ↩

  7. 1984 (George Orwell) ↩

  8. The 4-Hour Workweek , Escape the 9-5, Live Anywhere and Join the New Rich (Timothy Ferriss) ↩

  9. Deep Work (Cal Newport) ↩

  10. No More Mr. Nice Guy (Robert A. Glover) ↩

  11. It Doesn’t Have to Be Crazy at Work (Jason Fried; David Heinemeier Hansson) ↩

  12. Man’s Search for Meaning (Viktor E. Frankl) ↩

  13. The Prophet (Kahlil Gibran) ↩

  14. The Death of Ivan Ilych (Lev Tolstoy) ↩

  15. The Brothers Karamazov (Fyodor Dostoyevsky) ↩

  16. The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Change (Camille Fournier) ↩

  17. The Five Dysfunctions of a Team, Enhanced Edition: A Leadership Fable (J-B Lencioni Series) (Patrick M. Lencioni) ↩

The article’s thumbnail image background was created via Open AI’s DALL-E, with the prompt: “bright, lively and colorful epic light coming from piles of books, with pages flying, against white background, digital art”

]]> 2022-12-29T00:00:00+00:00 Learnings as a Software Engineer Techlead https://lopespm.com/2022/03/22/learnings-software-engineer-techlead.html https://lopespm.com/2022/03/22/learnings-software-engineer-techlead.html

YouTube video with a summary of this article

Life is growth. You grow or you die.

― Phil Knight, Shoe Dog

I’ve had the good fortune of having the opportunity to develop my software engineering work as a tech lead (which is not a fixed role), focusing more of my effort on solving sets of problems, as opposed to individual projects/tasks. While navigating this path, I’ve gathered a set of teachings from my mentors/peers/managers and other personal learnings and observations that I attempted to compile in a condensed list. Hopefully these can be useful in your own journey, just like other people’s learnings were to mine:

  • Planning:
    • Clear planning and prioritization of your projects is essential, especially in the early phases of an initiative. Well defined projects, with solid goaling metrics and well defined ownerships for each project, go a long way towards setting them up for success. These reduce the risk of not meeting goals, and allow for everyone involved to know which areas they need to tackle and own. Another interesting side effect is the value proposition to prospecting new-comers to support your projects, since there is a clear definition of what they can own and how valuable their contribution will be to the team/organization.
    • But don’t plan too much - healthy balance is the secret: if your organization is fast moving and dynamic, excessive planning can have the deleterious effect that little execution is actually done, since most of the time is spent planning.
  • Design Docs, Strategy and Vision:
    • Design Docs are a valuable vehicle to crystalize how a complex system or process can be implemented, they incentivize the owner to structure their thoughts, are made stronger by receiving feedback from peers, and serve as an exceptional piece of documentation that will not only explain how a given system/process works, but also serve a rationale on why they were built that way. Here is an example template from Google
    • Strategy and Vision - This reflection will be mostly based on this great StaffEng article, which I vividly recommend. If you realize that you’ve rehashed the same discussion three or four times, look into your organization’s objectives, find the commonalities from five of your organization’s design docs, combine all of these together, and you have material to write a strategy. When the future’s too hazy to identify investments worth making, grab five strategies, combine them with your organization’s objectives and write a vision for two to three years out.
  • Scaling oneself: The best way to increase impact is by scaling oneself. I’ve personally encountered my speed limit when trying to solve a critical-path project, while at the same attempting to set the direction for the team. As much as I worked, placed hours, or made the process effective, I was the bottleneck. I’ve hit a speed limit, and at the time it was made clear by my manager that in order to increase impact, unblock our team, achieve our objectives and have a balanced workload, I would need to scale myself.
    • Scaling through Delegation - the 75/25 rule:
      • If you are ~75% certain that someone will correctly execute a given project/task, then delegate. The other ~25% is the unknown space that the person needs in order to learn, explore, make errors and innovate. In other words, grow.
      • If you are 100% certain that the delegatee will successfully achieve a given project/task, they will not grow. On the other hand, if the probability of success is too low, then the project/task will be frustrating, probably will not be completed, and will jeopardize the team’s objectives.
    • By default, don’t micromanage:
      • When delegating a task/project, give enough space for the delegatee to own it. In general, the objective is to clearly define expectations, clearly define areas of ownership, and agree upon which details you should be kept up to date, which meetings you should attend, and in which situations you would expect to be pinged for escalations.
      • Micromanaging only makes sense in very specific situations, such as when there is a high possibility of a major failure and help is needed to firefight and course correct.
  • If you are not failing, you’re not pushing hard enough. And probably not growing: Conversely, the 75/25 rule can be applied to oneself. If you find you are easily executing your projects with a high level of predictability, this provides a hint that you should look for bigger challenges and responsibilities. In other words, aim for a growth mindset, rather than a fixed mindset.
    • Errors: Making errors is completely OK and expected. They are part of the ~25% above. Strive for them to happen only once, otherwise they will fail to serve their learning purpose.
  • Mentorship:
    • Give: one of the best ways to learn how to support projects is to mentor a new joiner. This will give you the opportunity to develop your interpersonal and project managing skills as you support/scope/prioritize projects and tasks for your mentee, and will help you consolidate your knowledge about the team and its position in the company as you transmit this knowledge forward. This relationship is especially rewarding in the long term as you work together on other projects down the line.
    • Receive: having good mentors is one of the best ways to quickly grow and learn. I have had the good fortune to learn from specially good mentors throughout my tenure, who apart from sharing their deep technical knowledge about our systems and processes, provided me solid foundations (either directly or by example) on how to navigate ambiguity and tackle challenging projects/situations. Every new mentor will have something novel to share and teach to you, it is an ongoing process. Good mentors provide you a golden path that you can follow to cut through the exhausting search for viable solutions to a given problem.
  • Learn from others:
    • Outside your organization: other people’s experiences are immensely helpful, specially the ones that have been time tested and are well structured. Books are a good source of this knowledge, and I would recommend The Manager’s Path, from which I extracted many of my base understandings of what a techlead should be, do and behave. The book goes straight to the point, gives actionable examples and was written by a very experienced manager.
    • Reach out to your peers: specially in the beginning of my role as techlead, I was struggling to find the right balance between personal impact from my own actions and (the less concrete) directional impact. To resolve my conundrum, I’ve reached out for support from my manager, my original mentor and a fellow experienced techlead. Their experiences and advices helped me to tackle this subject, and other challenges that came along the way.
  • Measure your success:
    • Goaling against quantifiable metrics = solid prioritization and justification: Make sure that all your projects are goaled against an objective, quantifiable metric. Otherwise it will be hard to prioritize between them and justify their value during planning and execution to your peers and external stakeholders.
    • Clear and quantifiable goals = increased motivation: Projects with goals based on solid metrics allow for anyone working on a project to know how valuable their contribution will be. I would claim that working on projects with an ambiguous value proposition is frustrating and will receive less attention and effort from everyone involved, since it will not be clear how impactful their actions will be.
    • Measure your personal progress by asking for candid feedback: if your manager and/peers don’t give you candid feedback on your progress, ask for it explicitly. It is one of the best ways to measure your progress on your role and surfacing any blindspots.
    • No measurement = flying blind
  • Communication - what you transmit, and how, matters:
    • Written and oral communication is essential when explaining the rationale behind an objective/project/task, and when coordinating with your peers and other stakeholders. This is a skill that can be learned, and one that I am still actively evolving. The best way to improve it, as I have found, is to write more, speak more, make more presentations, and study the communication methods from others.
    • As a rule of thumb, most communication styles benefit from a “news article” structure: first a TLDR (Too Long Didn’t Read), then an overall context of the problem in question and its impact, followed by details such as the delineation of a set of solutions, ending with a call to action or further questions.
  • Imposter syndrome: this might stem from a mismatch between your perceived performance and expectations for your role. Work with your manager to explicitly define expectations. Ask your manager and peers for explicit feedback, so you can better calibrate your perceptions. If you feel like an imposter, odds are, you aren’t.
  • Take care of your body and mind, they are your ultimate tools: Last but not least, exercise, eat healthy food, nourish your relationships and strive for quality sleep. These are the essential tenants for the infrastructure (you) that will sustain all of the above points.
]]> 2022-03-22T00:00:00+00:00 Hacker News Daily https://lopespm.com/2020/12/25/hackernews-daily.html https://lopespm.com/2020/12/25/hackernews-daily.html This article will dive into the system behind the generation of Hacker News Daily, an open source lightweight daily Hacker News best stories website, with screenshots and top comments.


Motivation

Hacker News is my go-to source for relevant, interesting and constructive discussions on a wide range of topics. I usually consume it via Daemonology’s Hacker News Daily, to catch up on the most active topics in the community.

Daemonology’s Hacker News Daily presents the title, story link, Hacker News discussion link, and is optimized for desktop. I usually consult it on mobile, and when I am several days behind, I sweep through the archives and open several story/discussion links in separate mobile tabs, in order to triage the stories with a quick glance at their web page and their discussion’s top comments.

This project aims to ease that process, by generating a set of web pages presenting the best daily Hacker News for mobile or desktop, with screenshots and top comments for each story, while aiming to have a low footprint to the end user.

Architecture

Information flow diagram between the different components of the system

The basilar idea is to recurrently get the best Hacker News stories via its API; take screenshots of the web pages that these stories link to; and have this workflow executed via GitHub Actions.

In specific, these are the steps taken to generate the final web pages (illustrated by the above diagram):

1. Every day, a new Github Workflow is spawned, kickstarting the entire process.

2. Update History

  • The first step is to get the past days best stories from the last successful run, and add the current best stories by reaching out to the Hacker News API to fetch the current list of best stories, removing the ones that were already featured, and adding the top ones to the current history
    • To persist the past best stories history, an days_history.dat artifact is created in every run, which is a simple Python Pickle containing the story ids from the past days. These days are stored in form of a deque, in order to pop the older days as the history grows larger
    • The final result of this step is a new days_history.dat, which will be used in the next step below, and will also serve as a base for the next workflow
    • If a viable days_history.dat artifact cannot not be found in the previous successful runs, it is rebuilt by parsing the Daemonology’s Hacker News Daily web page

3. Create Day/Story Models

  • From the story ids provided by the previously built days_history.dat, a list of hydrated days with their respective stories are built. These models will posteriorly provide all the information needed to create the final web page views
    • Each story is composed of an ID, title, story link, Hacker News discussion link, and its top comments. This information is obtained via the Hacker News API (example call)

4. Create the generated folder. This is where the generated web pages and screenshots will be placed, in order to be later deployed to GitHub pages

5. Gather Screenshots

  • Using pyppeteer, an headless browser is created to navigate through all the story links
    • A screenshot is taken for each of these pages (after attempting to dismiss a possible “Allow Cookies” prompt), and stored as PNGs in the generated folder
    • Each of these screenshots are re-encoded in WebP, which allows for a smaller image footprint in browser’s that support it
  • If there is an unexpected error while processing the screenshots, it will not halt the overall workflow, since the final web pages can still function without screenshot images

6. Generate Web Pages

  • Jinja, a templating engine for Python, is used to define and generate each of the final web pages. Four web pages are generated from a single template. These four variants are permutations of either presenting all past 10 days or not, and either showing images or not:
    • Latest day, with images
    • Latest day, without images
    • Past 10 days, with images
    • Past 10 days, without images
  • It was chosen to keep these in separate static web pages, in order to keep them simple, static, and without additional dynamic logic

7. The generated folder is deployed to the gh-pages branch, which is deployed as a GitHub page, making these generated contents publicly accessible

Source Code

The full source code is accessible at https://github.com/lopespm/hackernews_daily and the generated website at https://lopespm.github.io/hackernews_daily, feel free to improve it or to leave some feedback.

]]> 2020-12-25T00:00:00+00:00 Banishing mobile phones from bedroom http://lopespm.github.io/notes/2024/04/01/banishing_mobile_phones_from_bedroom.html http://lopespm.github.io/notes/2024/04/01/banishing_mobile_phones_from_bedroom.html 手机、闹钟、笔记、替代品、影响 总结：

文章讨论了作者如何通过更换闹钟和笔记方式来改善生活习惯，摆脱手机带来的干扰，提高专注力和生产力。作者选择了Braun闹钟和Onyx BOOX Note Air3 B/W作为替代品，取得了显著效果，更加专注于计划和思考，提升了日常工作效率。这种改变让作者更加满意，建议读者尝试。

The Problem / Why

When asked about which practices supported Chris Williamson into his current position, Chris mentioned that sleeping with his phone outside of the bedroom and getting a plain alarm clock was his number one change.

I found that statement interesting, especially because I had grown a habit of seeing the occasional video before sleep and being engaged with random distractions shortly after waking up.

Overriding these behaviors required a good amount of self discipline and willpower to be overridden, and fundamentally deviated my focus from what I would want to accomplish on a given day, from the book I had been putting off, or to simply wind down.

Alternative Alarm and Note Taking

Even though my phone sponsored my distractions, it was also my trusty alarm and favorite way to write down notes and reflections that crossed my mind outside my work desk. So to make the leap of keeping my mobile phone at the bedroom door, I would need to replace the alarm clock, and find a new way to write down my notes.

Alarm

This seems like an easy step, but it turns out the selection of alarms is immense, and many of them either do too much, or don’t tick all the boxes.

Essentially, I needed a trustworthy, low maintenance, alarm with essential functionalities that would not flood my dark bedroom with blue light when I wanted to check the time during night.

I essentially browsed through:

  • Lumie alarms: had the enticing proposition of science-backed approach of waking up via warm light or soothing sounds, but didn’t have the basic functionality of having recurrent alarms on the models I was interested in, and required a mains connection, meaning that upon a power outage it could only support itself for a limited amount of time.
  • White brand alarm clocks: the selection is immense on Amazon, and even though they come in at relatively cheap prices, some require a mains connection, or have short battery life, or need to be recharged every so often, or do too much (like having a bluetooth connection) or use noisy clock dials.

After several searches, I’ve stumbled upon Braun alarm clocks. These have been around for decades, and I recalled that Steve Jobs was inspired more than once by products made by Braun when creating his Apple products, due to Braun’s thoughtful approach on how everyday products were designed.

I’ve settled with the Braun BC08B. No moving parts, a single AAA battery apparently lasts more than half a year, small, simple, easy to operate, and the beeping sound is not brain drilling. The one aspect I am not fully satisfied is its screen light, which when activated is too strong on a dark bedroom (my mobile phone far excels in this aspect, due to its less intrusive always on black and white display).


Note Taking

These were the alternatives I found for writing down notes, and jotting down diagrams (as a plus):

  • Physical Notebook:
    • Pros: Easy to use, low cost, no distractions, low maintenance (no battery to recharge, software updates, etc)
    • Cons: My handwriting is appalling and my character output is much lower when compared to typing; extra step needed to transfer it a digital means; harder to organize
  • E-Ink Devices:
    • Remarkable 2:
      • Pros: well thought out, clean and focus driven, great writing experience; allows for handwriting to text conversion
      • Cons: Able to connect to a Google Drive for free, but works storage rather than syncing, so it is cumbersome to iterate on notes and diagrams between it and other devices
    • Supernote:
      • Pros: clean and focus driven, great writing experience; allows for handwriting to text conversion; open roadmap and ideology; easy repair and upgrade
      • Cons: As with Remarkable, it does not provide real file syncing, but file copy to Google Drive
    • Kindle Scribe:
      • Pros: Writing experience seems to be good and on-device handwriting recognition to convert notes into actual text
      • Cons: No Google Play apps, meaning that Google Keep and Google Docs would not be available, and no Google Drive sync; the device caters more to readers, rather than writers in my opinion
    • Onyx BOOX Note Air3 B/W:
      • Pros: it can use Google Keep and Google Docs apps, since can do almost everything a normal Android table can, but since it uses a e-ink screen and has limited processing capabilities, it does not make a good fit for browsing the web or seeing videos (which is actually a good thing); the default experience and drawing app are bloated offer too many distractions (see here how to debloat it); allows for handwriting to text conversion within Onyx drawing app
      • Cons: noticeable lag when writing; if not using Onyx’s cloud, it requires extra steps to export and share a diagram / drawing
  • iPad Pro: even though this device is amazing, it made it too easy to access the same distractions I was trying to avoid, and it also sports an invasive backlit screen that provides too much stimulus while in bed. A hard no.

Given the options above, I picked the Onyx BOOX Note Air3 B/W. Note that the color screen version of this tablet is available, but I did not pick it because of Air C’s reduced battery life from the always-on BSR (Boox Super Refresh Technology). BSR allows for a smoother and more responsive experience when writing for example, but that is not actually a problem since I mainly use the Boox to write long streams of text / braindumps, rather than text editing; and BSR also takes it a step closer to a general purpose tablet non-eink display, which starts to defeat the initial purpose. Apart from that, I have a personal preference towards the black and white screen feel.

Learn more on how I customized my Boox Air3 here.


The Result

I have now transitioned to not having any mobile phones in my bedroom, and only having the Boox Air 3, the Braun alarm clock, physical books and my Kindle. I am pretty satisfied with this change, which I’ve noticed to provide more focus time after waking up, more time to think, plan my day, reflect, write my thoughts, read, and wind down.

The most fundamental aspect was having more time to focus on my plan for the day, which translated into a substantial improvement on how productive and consequential my days have become. I welcome anyone to give it a try!

]]> 2024-04-01T00:00:00+01:00 Minimalist Onyx BOOX Note Air3 B/W Setup http://lopespm.github.io/notes/2024/03/25/minimalist_boox.html http://lopespm.github.io/notes/2024/03/25/minimalist_boox.html BOOX Note Air3 B/W, distractions, Lawnchair Launcher, bloatware, external keyboard
BOOX Note Air3 B/W被用于减少干扰，安装Lawnchair Launcher来提高效率，禁用多余应用以减少系统负担，使用外接键盘便于书写，效果显著。总结: BOOX Note Air3 B/W通过降低干扰，优化系统，外接键盘等方式提高工作效率，使得专注于书写成为可能。

As part of the plan to banish mobile phones with access to instant rewards from my bedroom (more on this in an upcoming post), I’ve acquired a Onyx BOOX Note Air3 B/W, which is a Android tablet with a black and white e-ink screen that is focused on writing and reading 1

House Rules

First of all, why I did get the device in the first place? This was in order to:

  • Reduce distractions before going to sleep and after waking up, increase focus on the things that really mattered, block out all other noise, and make it really hard to have access to instant gratification and rewards (such as video consumption)
  • Register any ideas and thoughts that occur to me before and after going to sleep, which is one of my most proficuous times for such, and then have them available on other devices

How to achieve this using the BOOX:

  • Only two functionalities should be easy and quick to use:
    • Write text and notes (via Google Keep and Google Docs)
    • Write diagrams and mental maps (via the device’s main app, which has the snappiest performance)
  • Getting to above functionalities should be as fast as possible, and they should be fast
  • The cost to get to other apps and functionalities should be high

Focusing & Debloating

The Onyx BOOX Note Air3 B/W is not an incredibly fast device, and comes with several functionalities, apps, and a launcher that increase the distraction degree, and also slow down the device.

Google Play, Google Keep & Docs

The first step towards the objective above was to install Google Play (see guide here, and afterwards installed these apps:

  • Google Keep
  • Google Docs
  • Android Hidden Settings - this app provides access to several hidden settings, that allow the access of settings such as enabling the Developer Options in Android (useful in the debloating process described below)
  • Lawnchair (see below)

New Launcher: Lawnchair

Before: Default BOOX Launcher 2

After: Using Lawnchair Launcher

The default device’s launcher was replaced by Lawnchair, which is a lightweight, open-source, highly customizable launcher for Android.

What was added to the Lawnchair’s home screen:

  • Shortcut to Google Keep
  • Shortcut to Google Docs
  • Shortcut to Onyx Launcher - for any custom changes that are only accessible on the device’s default launcher, which are are in practice, so this was eventually something I removed from the home screen as I wrote this note
  • Shortcut to Google Calendar - to add any events or reminders
  • A widget to Onyx Sketch app, which is one the only way I found to have that app directly accessible from the Lawnchair home screen, without having to use Onyx’s launcher to do so. The widget can be added by long pressing on the home screen, then select “Widgets”, and then on the “ContentBrowser” section select one of the “Notes” widgets

Disabling bloatware

With all of the above in place, we could start disabling non-essential apps and services, to prevent them from running in the background.

Here are the steps:

  1. The first step is to enable “Developer Options”, by accessing the system settings via the “Android Hidden Settings” app above, and then finding the Build Number in order to click it enough times to eventually enable “Developer Options”.
  2. Connect the BOOX to the computer
  3. On the computer’s terminal run the below commands

In general, these are the commands one can run via the terminal to manage bloat apps:

// See all packages
$ adb shell pm list packages

// See all disabled packages
$ adb shell pm list packages -d

// Disable a package
$ adb shell pm disable-user --user 0 <package_to_disable>

// Re-enable a package
$ adb shell pm enable <package_to_disable>


And these are the specific ones I used in my BOOX to disable bloatware, without any relevant side effects for my usages:

adb shell pm disable-user --user 0 com.onyx.appmarket
adb shell pm disable-user --user 0 com.onyx.floatingbutton
adb shell pm disable-user --user 0 com.onyx.dict
adb shell pm disable-user --user 0 com.onyx.mail
adb shell pm disable-user --user 0 com.onyx.igetshop
adb shell pm disable-user --user 0 com.onyx.easytransfer
adb shell pm disable-user --user 0 com.onyx.kime
adb shell pm disable-user --user 0 com.onyx.android.ksync
adb shell pm disable-user --user 0 com.onyx.calculator


The commands above should be enough to remove all bloatware, and if you wish to learn more, this article and this article provide a good overview.

Using an external keyboard

Given that this device is meant to be used in bed, and since it is quite unwieldy to write on the on-screen display, I was looking for an external keyboard that could be somehow attached to the BOOX, that would bestable, low-maintenance and allowed for flexibility on the writing position.

From the keyboards and keyboard stands I’ve researched, the most elegant solution I found was to use Logitech K480, a bluetooth keyboard with an in-built cradle, detachable batteries (i.e. zero issues related internal battery becoming faulty), incredible battery span (some users reported this to be in the order of years), and wonderful typing experience.

One caveat is that the BOOX slides backward and falls when using the cradle without any other aid. A very simple fix is to get a piece of cardboard, and place it behind the BOOX (see picture below) to create a better fit. It works remarkably well.


Writing and Drawing

Writing

Writing is my main use case for the device, and even though it has some lag when doing it, it works great for me, because I tend to write long streams of ranting text, rather than editing it.


The best mode I’ve found for writing was the “A2” Mode. It has ghosting, but is responsive enough for this use case.


Drawing

To draw, I use the default Onyx’s Sketch app due to its responsiveness, even though I would have preferred to have a more customizable UI interface and faster ways to share the drawing.

Sharing is one of the downsides of the Onyx Sketch app. You need to either use their cloud service, or export the image in order to get the sketch easily accessible on other devices. I’ve opted to use the export functionality, which allows to directly share the image to a Google Keep note, which fits well into my workflow. I would prefer to not have this extra step, but still, it works well enough.


Closing Thoughts

This has been one of the most satisfying devices I’ve added to my routines, and it’s incredibly satisfying to write on this device, almost distraction free. It allows enough customizability for it to be tailored to my use case, but is also limited enough to deter me from falling prey to distractions.

I hope this was useful towards your journey - do let me know your tips for distraction free environments!



  1. The BOOX Note Air3 B/W is a very capable device, able to do most of the things one would expect from an Android Tablet, but its screen refresh rate, speed and screen drag make it less suited for fast browsing and video usage for example. ↩

  2. The screenshot is from BOOX Tablet Note Air 3 C, the color counterpart of Onyx BOOX Note Air3 B/W, but the layout is the same, save from the color display ↩

]]> 2024-03-25T00:00:00+00:00 Musk's biography: why it's interesting http://lopespm.github.io/notes/2024/02/18/elon_musk_biography.html http://lopespm.github.io/notes/2024/02/18/elon_musk_biography.html 生物图书，埃隆·马斯克，成就，关系，幽默
埃隆·马斯克的传记展示了他的生活、个人特质、关系和幽默。他的成就展示了不可能是可以实现的，激励着人们去创造和实现目标。然而，这一切并非没有成本，马斯克承受了巨大的痛苦、风险和压力。传记中深入探讨了塑造马斯克的关系以及他对产品质量、设计和制造的不懈追求。同时，书中也揭示了马斯克的幽默感，以及他的"算法"——质疑需求、删除无用部分、简化优化、加快周期时间、自动化。作者深入细节地描绘了马斯克的故事，提供了丰富的数据点，为读者提供了有价值的启示。推荐这本传记给任何人阅读。

总结:

I’ve been hearing the latest Elon Musk biography 1 on audiobook, and just like Ashlee Vance’s book1, I’m finding it incredibly interesting. Here’s why:

Possibilities, Monty Python, costs, whys and whos

The action on potential

I find his path inspiring. It makes me want to create and do things, and shows that many things that are thought to be impossible, are actually achievable. There is a whole realm of opportunities waiting to be grabbed, and his life demonstrates just that.

There is a cost though. If it were that easy, everyone would do it.

The defining characteristics

The book goes deeper into the unique aspects that allow Musk to be an extreme overachiever, but also the personal and professional costs those same aspects inflict.

Musk’s capacity to take pain, risk, stress and adversity is staggering. I’ve been reading several biographies over the past years and met quite a few people throughout my life. I can only find a handful of them that are even comparable.

The relentless pursuit, sometimes maniacal, of product quality, functional and non-wasteful design, and effective manufacturing. All in the name of a greater objective, larger than any single human being or corporation.

The relationships

The book dives into the relationships that shaped Musk (for the better, for the worse, or both), the types of relationships he consistently attracted (which resonated with his personality and upbringing), and essential relationships that allowed him and his projects to thrive.

They come to further prove that no man or woman is an island. You can’t succeed by your own.

The humor

Consistently throughout Musk’s life, there is a balance between dark and light, between lighthearted comedy and downright hell.

I’ve had a good laugh with some of the humorous sides though, such as when eels and hovercrafts were catered to one of his marriages, or when Blue Origin filed a formal protest against SpaceX to prevent the company from having exclusive use of a NASA launchpad, to which Musk replied that “if they do somehow show up in the next 5 years with a vehicle qualified to NASA’s human rating standards that can dock with the Space Station, which is what Pad 39A is meant to do, we will gladly accommodate their needs. Frankly, I think we are more likely to discover unicorns dancing in the flame duct.”

Shortly after, a SpaceX employee bought dozens of inflatable unicorns and took a picture of them standing in a flame duct 🦄

Musk’s algorithm

“The Algorithm”, is a distillation of lessons learned while relentlessly increasing production capacity, which Musk repeatedly preached on his enterprises:

  1. Question every requirement
  2. Delete any part of the process you can
  3. Simplify and optimize
  4. Accelerate cycle time
  5. Automate

1. Questioning requirements

It can be summarized into this: “The only rules are the ones dictated by the laws of physics. Everything else is a recommendation.”. It’s a recurrent theme.

Doesn’t come without a cost though: it is often associated with conflict and chaos.

About 2. Delete any part of the process you can

“Delete, delete, delete”

Optimization many times goes hand in hand with deletion. Deleting is hard. It requires letting go of past achievements, comfort, and acceptance that something is not coming back. Almost like a breakup.

3. Simplify and optimize

Goes hand in hand with deletion.

4. Accelerate cycle time

This is where intensity and being “hard core” are most leveraged, in my view. Again, with their pros and cons.

5. Automate

A more obvious one. Put a machine to do it. Easier said than done though.

Data points -> cause & effect probabilities

I’m fond of biographies because they provide several data points that I can later use in my life. This is, which set of actions preceded a given consequence. Of course the context matters, and surely no same action will lead to the same consequence, specially if made by different people.

But patterns start to emerge, and their respective probabilities. These help me solidify my own personal theories and strategies. Learning from others is important. It allows for “shortcuts” similar to the ones provided by good mentors.

To build these, I need data points, a lot of them.

That is why I like Isaacson’s biographies so much 2. This is this third book I read (or hear) from Walter Isaacson, and one of the things I enjoy most about his writing is how deep he goes into the details of someone’s story. I believe that details are important when portraying someone’s life. Sometimes, the smallest of events make an immense difference over someone’s path.

Just read / hear it

Love him, or hate him, I recommend Musk’s biography to just about anyone.



  1. This link is an affiliate link, and as an Amazon Associate I earn from qualifying purchases whose commissions help this small establishment, at no additional cost to you↩

  2. Except Steve Jobs Biography 1, which I could not bear to finish. Jobs personality was a bit too much for me to handle↩

]]> 2024-02-18T00:00:00+00:00 Project Planning Framework http://lopespm.github.io/notes/2024/01/27/project_planning_framework.html http://lopespm.github.io/notes/2024/01/27/project_planning_framework.html 关键词: 框架, 目标, 时间敏感性, 项目依赖, 不确定性 总结:

本文介绍了一个简单的框架，用于将目标确定在三个轴定义的光谱中，从而确定实现目标的方法。作者提出的框架包括时间敏感性、目标/项目之间的依赖程度和步骤不确定性。接着作者分别用三个场景说明了如何应用该框架。第一个场景是合规性，时间敏感性高，依赖程度低，步骤不确定性低。第二个场景是增加用户流程效率，时间敏感性低，依赖程度低，步骤不确定性高。第三个场景是将多个系统统一为一个，时间敏感性低，依赖程度高，步骤不确定性中等。最后，作者强调了框架的简单性，并鼓励读者在使用框架时考虑可能的缺失参数。

Different goals require different roadmapping and planning techniques.

In this post, a simple framework is proposed to pin a goal into a spectrum defined by three axes, the combination of which informing the approach taken to reach the goal. The three axes of this framework are:

  • Timeline sensitivity: how much leeway is there to extend the deadline to deliver a goal?
  • Dependency level between goals / projects: how many steps are needed to reach the goal?
  • Uncertainty of which steps will drive the goal forward: how sure are we that a given step will move us toward the goal?

Example of how to apply the Framework

Scenario 1: Compliance

Goaling Statement

An external entity (let’s say, government or partner company), has made a set of requirements for which your team must provide compliance for on a fixed, non-negotiable timeline, with limited team capacity. The “definition of done” is well defined by this entity, and there is a low amount of unknowns of how to reach the goals.

Axes

  • Timeline sensitivity: High
  • Dependency level between goals / projects: Low
  • Uncertainty of which steps will drive the goal forward: Low

Approach

This goal has sensitive timelines and is archetypal top-down. The deadline will be reached in one way or the other, so all hands are needed on deck to tackle this goal.

Useful approaches could be:

  • Use a Gant-style chart, where there is a heavy emphasis on the timeline
  • Stricter project management, more process driven

Scenario 2: Increase Effectiveness of a User Flow

Goaling Statement

Your team supports a user flow, such as a check-out flow, for which the goal is the increase the conversion amount by X. There are known levers you have available to drive this metric, but also many others which are unbeknownst and would need experimentation and research. This increase is not business critical, so there is an underlying lenience in case the goal is not hit.

Axes

  • Timeline sensitivity: Low
  • Dependency level between goals / projects: Low
  • Uncertainty of which steps will drive the goal forward: High

Approach

Unlike Scenario 1, which is exploitation heavy, this scenario favors an approach which is exploration heavy, due to the high amount of uncertainty of which projects will be able to move the needle towards the successful completion of the goal. To de-risk and address this goal, a high level of adaptability will be needed, and several opportunities will need to be created to discover which levers can be used.

Useful approaches could be:

  • Have open ended brainstorming sessions
  • User research and focus groups
  • Looser project management, to allow for quick and less trashier pivots

Scenario 3: Unification of Several Systems into One

Goaling Statement

Your team is tasked to unify several internal systems into a single one, where several dependencies between different projects exist, and the time each project will take ranges from certain to unknown.

Axes

  • Timeline sensitivity: Low
  • Dependency level between goals / projects: High
  • Uncertainty of which steps will drive the goal forward: Low / Medium

Approach

The focus here should be on making clear which are the dependencies between different projects, and allow some leniency towards the completion of each of the projects, while actively managing the team’s capacity to focus on the projects which are discovered to require a heavier lift

Useful approaches could be:

  • A full dependency graph, allowing for each of the projects timelines to have some leeway where needed
  • Incentivize highly structured systems design documents and discussions, making sure that all involved parties are aligned on a given approach, and engineers are able to explain not only the system they own, but also the other systems involved in their unification

Closing Thoughts

The framework above is quite simple and could potentially be used for several other different use cases. It might also be missing some essential parameters that are missing for your specific goal. If you do find such, I would be happy to know. Regardless, hope the above can be useful on your next roadmapping cycle

]]> 2024-01-27T00:00:00+00:00 Make the meme transcend its host http://lopespm.github.io/notes/2023/12/31/meme-transcending-host.html http://lopespm.github.io/notes/2023/12/31/meme-transcending-host.html 关键词：ephemeral nature, digital persistence, physical medium, book, Benjamin Franklin 总结:
这篇文章讨论了数字内容的短暂性以及如何延长其寿命。作者提到了通过分配第三方或使用自持续数字机制来保持博客的在线存在，但这两种方法都有缺点。最终，作者推荐将内容保存到物理媒介，即书籍，以延长其存在的可能性。文章还提到了一些书籍对如何将想法转化成实质内容的启发，并分享了自己的计划，包括继续出版博客并将其编成书。

There is an ephemeral nature to everything. That includes me, you and this blog. This blog is currently hosted in GitHub, and nothing guarantees that after I’m long gone, or even before, GitHub might cease to exist. How to account for that?

One way to keep these writings alive, would be to assign a third person with the task of keeping this establishment online. What happens when that person is gone though? It’s turtles all the way down.

Another way would be to find a self sustainable digital mechanism to keep this blog alive, ie, a bot. That has its own issues as well, since technologies change and distributed systems like IPFS come and go.

The other option to extend the lifespan of these digital contents is to have these writings persisted into a physical medium. A book. No maintenance required. As long as it can be kept safe from unfortunate book burning events, the probability that a single copy remains alive, and can be made accessible (such as a public library) to someone and hopefully improve their life, increases.

No Input, No Output

The jump from a humble blog to a book is quite a leap. Outrageous even. Perhaps doable.

Some of the books I’ve recently read gave me some guidance and ideas on how to take that germinal idea into fruition (note that the following links are affiliate links, and as an Amazon Associate I earn from qualifying purchases whose commissions help this small establishment, at no additional cost to you):

  • Rolf Dobelli’s The Art of the Good Life is a collection of 52 articles that Dobelli wrote for a newspaper column.
  • Dobelli also previously wrote The Art of Thinking Clearly (I did not read this one), which reveals the most common errors of judgment, and how to avoid them, throughout 99 short chapters. Although the idea the idea of 99 chapters seems to have been original to Nassim Taleb’s project to publish “99 (convex) heuristics”
  • Steven Bartlett shares on his Diary of a CEO book that at 21 years old, he made a promise to himself that every day at 7PM, he would write a tweet or make a video delivering a single idea, and then post it online at 8PM, which Steven claims to have been the habit the most difference to advance his knowledge and skills. It completely changed his life trajectory, and consequently it’s the piece of advice Steven most strongly urges for anyone looking to be a better thinker, speaker, writer, or content creator. The key here was that he made it a daily obligation, not just an interest.
    • From this, I assume that this habit was foundational to his Diary of a CEO podcast, which eventually resulted in the book above
  • After listening to Benjamin Franklin’s biography on audiobook, I’ve came to learn that not only was Franklin a statesman, diplomat, scientist and inventor, but also a prolific writer, printer and publisher, which were unbeknownst to me. As a publisher, Benjamin Franklin wrote several dissertations and essays 1 (such as the ones under the pen name of Silence Dogood), which I’m sure to have fed into Franklin’s books, which brought many interesting ideas to a large population, like his own autobiography, which could be seen as a compilation of several of his ideas and reflections / writings (these go hand in hand).

Apart from finding the above points inspirational, they also make me think that creating a discipline where I consistently share my reflections and experiences via writing or videos, not only helps me solidify my thoughts and provides more immediate feedback on how valuable these are to its viewers / readers (which allows me to course correct and iterate 2), but also provides the building blocks to put together a hopefully meaningful and concise book to help surpass the limited lifespan of the digital medium that hosts this blog and my YouTube videos.

No Input, No Output

― Joe Strummer, lead singer of The Clash

The Path Towards Physical Persistence

Given the above, here are some action points to put the wheels in motion towards building “physical persistence” via a book, and give better chances for its memes to transcend their host 3:

  1. My low priority 2024 goal is continuing to publish blog notes and videos. The numeric goal I’ve set for myself is to publish 12 blog notes, about once per month. From these, make the respective YouTube videos, but this would be a stretch sub-goal (albeit, one that I really enjoy)
    • This was a consequence of my recent “personal performance” review for this year’s personal goals, which was a good opportunity to reflect on what went well, what went less well, what was achieved, and what was missed. One of the aspects that I aim to implement for the upcoming year’s goals is to use numeric goals, where applicable.
  2. Once 50 or 99 meaningful blog notes are written, attempt to compile them into a book.
    • Why 50 or 99? Upon reaching 50 notes, I might feel that not enough substantial content was generated to be distilled into a book. Once 99 notes are reached however, it would be more certain that enough meaningful content was accumulated (hopefully). I would expect the distillation process to remove several notes that would be deemed to have less quality.

Looking forward to 2024! 🎉



  1. On the extensive range of writings that Benjamin Franklin did, there are also hilarious examples such as Fart Proudly, an essay about flatulence written while he was living abroad as United States Ambassador to France↩

  2. One the learnings from making the Survival Ball video game, was that the lack of exposure throughout it’s development process lead to less visibility to the ones who might be interested in it, and less immediate feedback on what worked and what didn’t work, which could have possibly have helped to craft an better product, by virtue of allowing faster / less painful pivots↩

  3. Coined by the British evolutionist Richard Dawkins in his book The Selfish Gene (1976), a meme is a unit of culture—such as “tunes, ideas, catch‐phrases, clothes fashions, ways of making pots or building arches.” In humans, memes have supposedly taken over much of the evolutionary burden of the traditional units of heredity, the genes. Dawkins introduces them because in his opinion the rate of human cultural evolution is far too rapid to be simply a function of gene‐centered evolution.↩

]]> 2023-12-31T00:00:00+00:00 Job Safety in the AI Era http://lopespm.github.io/notes/2023/12/25/job_safety_in_ai_era.html http://lopespm.github.io/notes/2023/12/25/job_safety_in_ai_era.html 人工智能, 人类连接, 生物工程, 机器工具, 未来保障

人类将面临越来越智能的机器取代。然而，人类的情感和情感联结以及对现实世界复杂性的处理能力仍然是机器无法取代的优势。因此，人们应该努力接近人与人之间的联系，并专注于为人们或企业提供有形的价值。通过这种简单的思维方式，人们将能够获得丰富的（金钱、情感等）价值，并提供这些价值。

总结:

Words are cheap and keep getting cheaper. The last notes on this blog could have been written via generative AI, and as time goes by, new generative models will likely create higher quality content than I could ever would, consistently.

I didn’t use these because the writing process comes as a form of thinking, but if there are automated mechanisms that increasingly outperform us humans in so many different areas, what does that leave us?

Human connection

How comfortable would you be in having a relationship with a being that is orders of magnitude more intelligent and capable than you? Not just logical-mathematical intelligence, but emotional, linguistic, musical or even spatial intelligence.

A being so advanced that any form of communication or relationship between you and it would always be severely constrained, even with neural interfaces. A being so advanced that you would never be able to have an essential understanding of it. A being so advanced that you wouldn’t even know that you are being manipulated by it. How many humans do you know who are friends with chimpanzees or bonobos, our closest relatives?

We are still human beings, with all the perks and limitations that come with it.

I would claim that humans will always be attracted to their familiar counterparts. Other beings that share their own struggles and limitations, and by consequence, what they create and share. Machines have their important place in the world, and history has proven to us that life as whole improved as technology evolved to serve us, but it would be healthier for us to think of them as tools, rather than human replacements.

Bioengineered

We have been fitted with extraordinary senses and tools that allow us to efficiently interact with the world. It took milenea for nature to shape us, and as Leonardo da Vinci puts it when comparing natural to artificial structures: “Nature made the most perfect inventions. There is nothing superfluous, there is nothing lacking” (paraphrased)

Our place

Try having a machine fixing the leaky pipe in your bathroom, or to change your desk’s lightbulb. Odds are that you’ll be flooded and in the dark in the nearterm. There will come a point where they will likely do that, but we are still not there.

Machines / tools currently outperform us in several tasks on the digital realm, and in several heavy duty physical tasks. But when it comes to fine digital dexterity, human connection, and interaction with the real world and its complexities, we still have an edge.

Future Proofing

Take a look at the daily tasks in your job. How repetitive are they? If they are repetíve, how much of a human element is present? Look at your organization and see where rubber meets the road in terms of human connection and physical intervention.

Strive to be closer to that human interface and focus on delivering tangible value to people or businesses (group of people). With this simple mindset, I believe you’ll be in a safer place, surrounded with an abundant supply of meaningful (monetary, emotional, etc) value that you can extract and provide.

]]> 2023-12-25T00:00:00+00:00 New York City Guide http://lopespm.github.io/notes/2023/12/25/nyc.html http://lopespm.github.io/notes/2023/12/25/nyc.html Google Maps, Museums, Entertainment, Points of Interest, Transportation
这篇文章是一位游客对纽约的全面游记。作者分享了如何利用谷歌地图的事先准备，以及游览时的经验。他介绍了多个博物馆，包括平流层博物馆和大都会艺术博物馆。此外，他还探索了娱乐活动，如百老汇表演和喜剧俱乐部。文章还提到了一些值得游览的地方，如自由女神像和中央公园。最后，作者介绍了在纽约的交通方式，包括步行、渡轮和地铁。整体而言，这篇文章是一份详尽的纽约旅行指南和个人体验分享。

I’ve recently been to NYC. It was one of the most interesting experiences I’ve ever had.

Each one has their own perspectives when visiting the city, and I’ve asked several people about theirs before the trip, which helped immensely when planning out my itinerary. Now it’s my time to pay it forward. Here my guide / personal perspective of New York.

Maps

Having a pre-curated list of points of interest on Google Maps was one of the most useful and time efficient aspects of the entire trip. This allowed me to quickly improvise and make the most of my surroundings, since I could just look at the map to check which nearby places could I visit at any given time.

Museums

I didn’t consider myself to be an excessive photographer. I’ve discovered on this trip that I was probably wrong. I can’t recall the last time I took so many photos, it was like I was in an outstanding candy store of visual goodies.

Intrepid Museum

Let’s start with fashion. I’ve been pointed out that my glasses make me look nerdy / square / too serious / too much like an engineer. Well, thanks to this museum piece, I can now say they were developed for space walks:

Oh, and look back and you will also see the Space Shuttle Enterprise, the first orbiter of the Space Shuttle system:

You can find these glasses, the space shuttle, sit inside the Mercury capsule or A-6 Intruder cockpit, see real life jet planes like the Lockheed A-12, in the Intrepid museum.

Where is the Intrepid museum? The museum is an World War II–era aircraft carrier. Yes, the vessel is the museum. You can explore several of it’s interior sections, including the bridge, living quarters, gun and bomb bays, go through the same narrow escalator that pilots took to the flight deck, all of these throughout several floors and different explorable rooms

Right beside the Intrepid lies USS Growler, a diesel powered submarine retrofitted to deliver Regulus nuclear missiles, as a form of nuclear deterrence during Cold War. You can go inside the submarine and get an intimate experience of how it’s lived throughout their 2 month patrols, where they faced the constant scenario of being called upon to launch a nuclear missile onto Russian soil, destroying not only their target, but most likely kill themselves on the process, since the process to launching / reloading Regulus missiles was long and would expose this submarine to enemy reconnaissance.

  • Since it was a diesel submarine, it required to be at surface or near surface level (it was not possible to push out their exhaust gasses when deep in the ocean) to enable a set of its 3 engines to charge its on-board batteries that would power the submarine and its electric motors.
  • It also needed to be periodically refueled, so the submarine ran a long circular patrol in the Pacific Ocean to refuel to eventually sit near Russian enemy lines for dozens of days.
  • The crew had to maintain absolute secrecy about their mission, and none of their partners knew the context of it
  • The families of the crew members knew nothing about their mission, and that if they did launch a nuclear missile, they would be exposed, and most likely be killed as they expose themselves to launch the nuclear load (not to mention the annihilation that comes from the nuclear warhead)

I found the underlying story and details of this submarine and this mission were so interesting, that right after the tour, I went immediately in search of staff members to ask a series of questions that came about during the visit. They were very kindly and patiently took the time to answer them, and one of the ladies even asked me if I was an engineer. I’m not sure what gave that away. It was either the questions, or my glasses :)

MET

MET, or Metropolitan Museum of Art, has two locations: the museum in Fifth Avenue is the most famous one and is often referred to when mentioning MET. The other lesser known branch is in Upper Manhattan, the MET Cloisters.

MET: Fifth Avenue

  • If you like the British Museum, you will love MET on Fifth Avenue
  • They reconstructed the 10 BCE Temple of Dendur Roman Egyptian religious structure, which originally located in Tuzis (later Dendur). Must I say more?
  • This massive museum spans through classical antiquity and ancient Egypt; paintings and sculptures from nearly all the European masters; and an extensive collection of American and modern art. All of these are intertwined with large thematic rooms where the installations are beautiful to behold. If you are going to see a single museum in New York, this should be the one.

One of the last books I’ve listen on audiobook was Walter Isaacson’s Benjamin Franklin biography, where one of passages goes through the story of how a Duplessis’ painting of Franklyn’s portrait came to be. This 1778 painting can be seen in the museum, and curiously enough, Franklyn’s depiction on $100 bills from 1914 to 1990 had him wearing a fur coat, just like this painting.

MET: Cloisters

With the same ticket, one can visit the Cloisters and the 5th Avenue museum on the same day. The Cloisters museum is America’s only museum dedicated exclusively to the art and architecture of the Middle Ages, and is smaller than the 5th Avenue one. Still there much to explore not only inside the museum, but also on its surroundings

Surrounding the Cloisters is the beautiful Fort Tryon Park and it’s Heather Garden, sitting right next to the Hudson Riven. A bliss to behold.

National History Museum

  • I lost count of the amount of dinosaur skeletons I’ve seen in the natural history museum. And nothing quite prepares you to see a gigantic skeleton of a Titanosaur. It’s so big that its head extends outside of its new home in the Museum’s fourth-floor gallery.
  • I’ve learned while browsing the museum about mass dinosaur graves and their relationship to draughts, and went through thought provoking sections, such as the presentation of a massive turtle specimen, that if it were to be extinct it would probably be right part of our imaginary and event movies.

Another thought that stuck to me is how deeply connected we are to other animals, Earth and outer space. Seeing life sized representations of several of these makes them immediately more relatable.

Guggenheim

On Saturdays, from 5 to 8 pm, admission to the museum is “Pay What You Wish”, for a minimum of $1. I would recommend doing so, since the museum can be easily visited in less than one hour, and in my opinion, it’s more about the building than the pieces themselves

Entertainment

Broadway

Broadway shows are a hallmark of NYC, although if you are from London or Europe, I would recommend not seeing shows that are already available in London. For example, I’ve heard that Hamilton is better seen in London, due to the larger contextual part that King George has in the UK.

The Chicago musical is a good bet, and is the one I’ve attended to, it is the second longest-running show ever to run on Broadway, behind only The Phantom of the Opera

Comedy

Comedy Cellar is a comedy club in Manhattan where many top New York comedians perform, and where several comedians started, like Louis C.K. or Dave Chappelle. I found it be have down to earth, edgy comedy, and you are asked to leave your phone in a pouch, which I believe to increase presence of the entire crowd, but also provides extra freedom to comedians, given to how powerful cancel culture can be

Points of Interest

Statue of Liberty

You can get the Staten Island Ferry from the Whitehall terminal, for free. On your way to the terminal, several people will try to sell and cajole you into payed trips to the Statue of Liberty trips. The ferry is more than enough. - The ferry runs every 30 minutes, 24 hours per day, for most of the days, and travel time is approximately 25 minutes. - Once you get to Staten Island, you can either take some time to explore it, or just return immediately to Manhattan, by exiting the ferry, by entering St. George’s Ferry Terminal, and entering the boarding gates for the next ferry. - When entering the Whitehall terminal, enter through the right side of the ferry in order to be on the same side that the Statue of Liberty will appear. If entering on St. George’s Ferry Terminal, enter through the left side of the ferry. Regardless, you can easily change sides when inside the ferry - Most of the people will likely go to the top floor with larger windows, which will eventually get crowded. I recommend instead to take the bottom floor, and find a window that is already opened. The view is much clearer (since you don’t have a worn down piece of glass obfuscating the view), you can sit comfortably near the window, and you won’t need to fight a crowd (in principle)

Central Park

Central Park is an urban park between the Upper West Side and Upper East Side neighborhoods of Manhattan in New York City that was the first landscaped park in the United States. It is the sixth-largest park in the city, and it’s a great way to step out of the busyness of the city and discover the treasures that lie in the park.

High Line and Little Island

The High Line is a 1.45-mile-long elevated linear park, greenway and rail trail created on a former New York Central Railroad spur on the west side of Manhattan in New York City. If starting it from the north side, you can enter it near Hudson Yards, and walk your way south from there.

Upon the southern end of the High Line park, you can view Little Island, and quickly walk to it. Little Island is an artificial island park, with some beautiful views and interesting layout

Financial District

9/11 Memorial

The memorial is located at the World Trade Center site, the former location of the Twin Towers that were destroyed during the September 11 attacks. Each of the towers’ footprints are now the home of two large, recessed pools. The sheer dimension of these is awestrucking. The entire site is filled with symbolism.

Wall Street

  • On the street itself, we can find the New York Stock Exchange, the largest stock exchange in the world, which for decades filled the popular imagination with its hustle, bustle and hectic traders. Those same brokers and traders are now surrounded by computers that manage the majority of the buying and selling of stocks for their various accounts. Floor trading still exists, but it is responsible for a rapidly diminishing share of market activity.
  • The Charging Bull of Wall Street is not really located on Wall Street itself, but it is quite near. The bull in finance represents optimism and growth. The statue on Wall Street represents the same ideas.
  • The Federal Hall is a historic building, whose original building served as New York’s first City Hall and hosted the 1765 Stamp Act Congress before the American Revolution. With the establishment of the United States federal government in 1789, it was renamed Federal Hall, as it hosted the 1st Congress and was where George Washington was sworn in as the nation’s first president. The current structure was built as the U.S. Custom House for the Port of New York before serving as a Subtreasury building from 1862 to 1925, and the current national memorial commemorates the historic events that occurred at the previous structure.

Columbia University + Morningside Park

Walking west from Central Harlem, where you can find the Apollo Theatre for example, we reach Morningside Park, from which we can climb up towards Columbia university, a private Ivy League research university in New York City.

Grant’s Tomb

  • In a mere 7 minute walk from Columbia University, you can reach the General Grant National Memorial (Grant’s Tomb), classical domed mausoleum, the resting place of an American military officer and politician who served as the 18th president of the United States from 1869 to 1877. The underlying background music and the attention to detail are something to behold
  • Right beside it sits the soaring Riverside Church, all of this surrounded by the peaceful Riverside Park

Citigroup Center

Due to material changes during construction, the building as initially completed was structurally unsound. To save money, Bethlehem Steel changed the plans in 1974 to use bolted joints, and wind loads were calculated from perpendicular winds, as required under the building code; in typical buildings, loads from quartering winds at the corners would be less.

In June 1978, after an inquiry from an engineering student, the structural engineer recalculated the wind loads on the building with quartering winds, and found these to significantly increase the load at the bolted joints, and that a wind capable of toppling Citicorp Center would occur every 55 years on average.

Starting in August 1978, construction crews covertly fixed the issue, and six weeks into the work, a major storm was off Cape Hatteras and heading for New York. The reinforcement was only half-finished, with New York City hours away from emergency evacuation. The storm eventually turned eastward and veered out to sea. The repairs were finished successfully, and no major issues happened ever since.

The Woolworth Building

Completed in 1912, the Woolworth Building was the tallest building in the world from 1913 to 1930, with a height of 241 meters. It blows my mind that such a tall building was built more than 100 years ago.

Billionaires’ Row

Home of one of the most expensive apartments in the world, the Billionaires’ Row hosts a group of ultra-luxury residential skyscrapers. Several of them are so thin that it’s hard to believe how they can stand upright without toppling over.

Lincoln Center

  • The Lincoln Center is a complex of buildings in the Lincoln Square neighborhood on the Upper West Side of Manhattan, where the beautiful Metropolitan Opera House can found.
  • It houses internationally renowned performing arts organizations including the New York Philharmonic, the Metropolitan Opera, the New York City Ballet, the Chamber Music Society of Lincoln Center, and the Juilliard School.

The Bronx

The Joker Stairs

The first picture on this note was taken at the “Joker Stairs” is the colloquial name for a step street connecting Shakespeare and Anderson avenues at West 167th Street in the Highbridge neighborhood. The stairs are quite different from when Joaquin Phoenix danced as the Joker. They are more colorful and with a construction side on the top.

The Birthplace of Hip Hop

DJ Kool Herc is credited with helping to start hip hop and rap music at a house concert at 1520 Sedgwick Avenue, which is currently covered by scaffolds as seen here. I would recommend to go during daylight hours and to be attentive on your route as it gets rougher on the way there

Yankee Stadium

The stadium is the home field for the New York Yankees and New York City FC, and also has some small surrounding parks

Brooklyn

Williamsburg (Bridge)

While the Brooklyn Bridge stands as the most famous bridge to cross the East River, you can easily dodge its flocking crowds by crossing the Williamsburg Bridge, which leads you into Williamsbridge, characterized by a contemporary art scene, hipster culture, and vibrant nightlife that has projected its image internationally as a “Little Berlin”

You can find several interesting places in Williamsburg such as the Domino Park, Spoonbill & Sugartown Books and the City Reliquary Museum

Brooklyn Bridge

The Brooklyn Bridge is an iconic landmark of NYC, which you can cross from Manhattan towards Brooklyn. Be advised that this crossing is often quite crowded, and there are several vendors in the bridge itself

Dumbo & Brooklyn Heights

Once you cross the Brooklyn Bridge, you can head directly to Dumbo, where you have beautiful view to the Manhattan Bridge

You can also have several scenic views towards of the Brooklyn Bridge and Manhattan, in both Dumbo and Brooklyn Heights

Brooklyn Museum & Brooklyn Library

  • The Brooklyn Museum is New York City’s second largest and contains an art collection with around 500,000 objects, and is fairly easy to access from the subway. The breadth and depth of objects at the Brooklyn Museum is best compared to the Met, boasting strong departments of both Western and non-Western holdings. I didn’t fit enough time to visit the actual museum, but it is definitely on the todo list in case I visit the city again.
  • Brooklyn’s Library four stories high main entrance is a marvel to behold just by itself

Prospect Park

Prospect Park is an expansive and peaceful urban park with beautiful landmarks and structures, such as the Boathouse on the Lullwater, several watercourses, bridges, monuments and statues

5th Avenue

Rockefeller Center Christmas Tree

The Rockefeller Center Christmas Tree has been a yearly tradition ever since 1931, which now hosts a 20m+ Norway spruce, near to an also historical skating rink, which was opened below the tree in the plaza in 1936. This year, you’ll be able to see the tree up until January 13th 2024.

Saks

  • Directly facing the Rockefeller Center, you can find Saks Fifth Avenue, an American luxury department store founded in 1867.
  • In partnership with Dior, as part of Saks’ annual holiday, it is hosting a massive, bronze circle is broken up into eight sections with a deep blue center decorated with all the zodiac signs and a spray of stars, with one big star in the middle, where there is a periodic visual and sound show which is truly impressive.

Trump Tower

Just a few blocks north from the above places lies the Trump Tower, where Donald Trump descended on an escalator to announce his candidacy for president, the first step on a journey few believed would take him all the way to the White House.

Eat, Sleep, Move and Pay

Where to Eat

  • Unless you would be aiming to hit a specific high demand restaurant at a specific time span, there is limited need to pre-book a restaurant, since you can just walk for a few minutes and find other quality restaurants.
  • There are several food stalls around NYC, many of which you will be able to recognize by their LED letterings and smell of burnt meat, I would personally avoid these. There are food stalls which are incredible though. One of these is Shawarma Bay, just a block away from Radio City Music Hall. Incredible food, although it can gain quite a queue, which can take about 30 minutes, as it was my case. Worth the wait though– cops were eating there
  • You can get incredibly good Deli Sandwiches in NYC, in places such as Sunny & Annie’s Deli. I’ve personally tried one at Gold Deli, and it was seriously good.
  • Are you in Times Square and want to grab a bite? Walk 2 blocks west and you will find yourself in Hell’s Kitchen, packed with good restaurants, without much less hassle
  • Wan Wan Thai restaurant. That has got to have been the best dining experience I had in NYC, for an acceptable price.
  • Joe’s Pizza offers traditional New York slice-style pizza at a fair price, that has made a dent in popular culture. Definitely worth the it.

  • Williamsburg’s L’Industrie Pizzeria apparently has some of the best pizzas in NYC, but I can’t attest to that, given there was a massive queue when I arrived there. So I moved to Rosa’s Pizza, where I had delicious slice.

  • On markets such as the Chelsea Market you can find several good food places.

  • Fraunces Tavern is a museum and restaurant that played a prominent role in history before, during, and after the American Revolution. At various points in its history, Fraunces Tavern served as a headquarters for George Washington, a venue for peace negotiations with the British, and housing federal offices in the Early Republic.

Where to Stay

  • This all depends on your budget, activities you would like to do, who is going with you, etc. But in general, I would advise on staying in Manhattan island, or at least near to a transport connection that can get you quickly to the city center, since it is where most of the points of interest are located, and transport links are quite good from there. Most likely you will also be in your lodging to sleep, and not much else.
  • If looking for a hostel, HI New York City Hostel is a good one. It even sports free weekly comedy sessions every Sunday night. It’s located near Central Park, and you can’t go wrong with this one. In general, you can search through https://www.hostelworld.com/ and book through there, it was quite reliable for the two bookings I did.
  • If looking for something more comfortable, citizenM Hotels are of incredible good value and location. Apart from the comfortable rooms that have nice view already, on Bowery it even has rooftop terrace with an amazing view of the city

Money / How to Pay

  • Stores, food establishments (even chains), bodegas / delis, 7-11s, etc present their prices as pre-tax. New York City sales tax ranges from 4% to 8.875%, and some products are exempt from it.
  • Be prepared to tip everywhere. On restaurants it is customary to tip 20%. There is some debate about whether you should calculate the tip on the amount before or after tax.
  • You don’t really need cash. You do need it if you want to buy from street vendors, or if you want to dodge the additional cost that certain sellers impose on card transactions (like the shawarma above)
  • On restaurants and entertainment venues, it is common practice for the attendant to leave with the credit card, to later return with an attached invoice that is meant to be signed and filled with the tipping amount.
    • It also happened to me that when trying to pay by contactless card to a market vendor, the card kept being rejected, and since the card insertion feature was not available, the vendor wrote down the card number and details into the system to proceed with the payment. That failed as well, because the card got blocked by my bank due to a false-positive suspected fraud, as I later discovered. Eventually I just payed with cash to resolve the issue.
    • As an European, the practice of being physically separated from the card, verification via signature, and freely writing down credit card numbers strike me as quite unsecure.

Getting Around / Transportation

  • Walking is one of the best ways to discover the city. Sidewalks are broad, the city is grid shaped (so expect to be given coordinates in the form of street / avenue to pinpoint a place), and pedestrian lights and timings ran standard everywhere I visited. I did the large majority of my exploration of the city over the span of 180km throughout 4 days. Extra tip: if you are used to barefoot shoes, wearing firm ground outsole Vivos to do so is amazing. My feet were sore, but happy.
  • Walking through the perpendicular numbered streets and avenues is an absolute bliss, and makes it quite easy to navigate through them. Several times I just looked at the map and quickly memorized something like “3 right, 2 left”, which translated to “cross 3 roads and turn right, and then cross 2 roads and turn left”. This would keep me walking going for a long time and have time to appreciate the surroundings, without the need to constantly check the map.
  • Ferry boats are a great way to sight the city, for a low price ($4 for a one way-ticket), and you can buy these tickets through the NYC Ferry app or a ticket vending machine.
  • Subway and Bus now have OMNI, which just means that you can tap in with your credit / debit card. Each ride costs $2.90, and there are no different prices for different zones. This has a [rolling 7 day price cap] (https://omny.info/fares), so you never pay more than $34 over that period.
    • Metro Card is the pre-paid version of this. You need one, buy them directly from the machines to avoid being scammed. For example, a 7-Day Unlimited MetroCard costs $34 upfront, and can be used any number of times during those 7 days.
      • Be aware that if you swipe in, get out from the station, and then attempt to shortly swipe in again, it will reject your swipe, due to being too recent. If that happens, ask an MTA attendant to let you in (if available), or do like many others and wait for someone to crack open the emergency exit from the inside. If you hear an alarm going off in the station, most likely is because someone opened the emergency exit to let others in.
  • Just like routine events such as figuring out post-tax prices purchases or how much and when to tip require some kind of mental effort, figuring out the different edge cases of the subway system will keep your mind busy. It is like there is a perpetual rat race game being played in the city.
    • Sometimes trains stop at all stations, sometimes they don’t, sometimes trains have indicators of which station you are in, sometimes they don’t, sometimes the station has (non-standard) indications of how much time it takes for the next train to arrive, sometimes they don’t. Sometimes it rains heavily inside the stations, sometimes it doesn’t.

Other Notes

  • Something unexpected to me, which cannot be transmitted through visual media, is the collection of smells around New York, in almost all parts of the city, except a set of parks and neighborhoods mostly outside Manhattan. The smell of garbage (since it is comment that it accumulates in heaps of bags on city sidewalks), burnt street meat, horse manure in some parts of Central Park, a smell I can only describe as pervasive detergent / laundry smell that could even be detected inside several buildings like the MET and Delis; the strong smell of road paint (since it appears that several of the streets were recently repainted), and signature smells from the subway
  • Several subway lines are relatively shallow, so they are easily heard at street level. They also generally run below the roads themselves, which I would assume to make it easier when building incredibly high skyscrapers whose foundations run quite deep.
  • Be attentive of your surroundings and where you walk, such dodging through garbage debris and sidewalk cellar doors, be them open or closed
  • Be prepared to encounter people who are not as well off, and might be living in the streets. Use your judgment, keep out of confusions, be respectable and don’t engage with people who have serious mental issues. If you are in a rougher area, memorize your path when possible, and stop at a local shop if you need to get your bearings right. Use your common sense
  • You might have noticed that I didn’t mention the common Edge / Top of the Rock / Summit One / Empire State Building / etc observation decks. It’s not that I don’t enjoy incredible views, but it didn’t fall high enough on the list of priorities, specially because I’ve had the opportunity to have skyline views through other buildings I’ve passed by, such as the Hotel.
  • I tend to like long trans-atlantic flights, since they provide a moment of low stimuli, where there is less incentive to use the internet. My personal piece of advice would be to avoid the usual in-flight entertainment. I love to use them to resume a book or to write down my reflections from a trip for example (several of article lines were written during the return flight)
  • This only shows my ignorance, but I had no idea that so many well-known artists were originally from New York, like Jay-Z (Brooklyn), Cardi B (South Bronx), Spike Lee (Brooklyn), and of course, Jenny from the block. That explains why I’ve consecutively heard “Empire State of Mind”, from Jay-Z and Alicia Keys (Hell’s Kitchen), and “All I want for Christmas is You”, from Mariah Carey (raised in Huntington).

Closing Thoughts

Ever since I was a kid I had considerable fascination with the USA. The culture that arrived through the small and big screens, the big companies and entrepreneurs that I admired, the language, the people I’ve met along the way, the stories. Having had the opportunity to visit it and experience first hand was a true privilege. I’ve lived it like it was the last time I would visit it, and it imprinted in me several thoughts and perspectives that I’m sure will last a lifetime.

New York ain’t no DisneyLand, and Elmo’s pictures in Times Square don’t come for free either, so set expectations accordingly. It can be one of the most fulfilling experiences you’ll ever have. I advise you to come prepared and plan accordingly.

Frank Sinatra sang that he wants “to wake up in the city that doesn’t sleep”. That phrase has a deep meaning that only struck me upon my trip. The city does not indeed sleep. Times Square lights, 24 hour subway, the constant movement. Living in NYC does wake you up, not only literally, but also figuratively. It wakes you up towards a different reality that keeps you on your toes in several ways. In a way, it’s a celebration of life.

]]> 2023-12-25T00:00:00+00:00 Relative Cost / Benefit of an Investment http://lopespm.github.io/notes/2023/10/21/investment_relative_cost_benefit.html http://lopespm.github.io/notes/2023/10/21/investment_relative_cost_benefit.html 投资成本, 使用率, 产出, 比较, 规则简化 总结: 这篇文章通过简化的例子，介绍了投资成本与使用率之间的关系，并将其应用到不同的场景中。关键点是，一项投资如果闲置不用，将会产生较高的相关成本。文章提供了一些例子，如租房、学习工作、购买衣物等，通过计算每小时或每次使用的成本，展示了投资的成本效益。这种简化的启发式方法可以在不同的情况下作为一个做决策的参考。文章还提到了一些额外的成本和利益，如其他直接和间接成本，以及可能的资产回收等，以及这些例子与初衷的对比和期望。虽然这种简化没有考虑到所有的因素，但可以帮助我们更好地评估投资所带来的效益。

Let’s say you spend $1,000 to acquire film equipment (camera, microphones, and lights) to shoot YouTube videos. If you use this equipment to film and publish 2 videos, it means that each of those videos costs $500 to produce. If 100 videos are published, each of them cost $10. If all of these videos are able to generate $10 of revenue, then the entire cost of the initial investment was amortized.

The above is a severely gross simplification1, but the key point is that an investment left unused has a high associated cost. In the case above, if only two videos were to be produced, then it would have been a better idea to rent the equipment instead.

This simple heuristic of mapping the investment cost to its usage and upside, can be a potential rule of thumb in different scenarios.

Other Examples

  • Apartment Rent
    • You are renting an apartment for $1,000 per month. During the week you work in an office, on weekends you are sometimes out, and during the year you take sporadic vacations outside. Let’s say that, on average, you spend 14 hours per day on your apartment, including sleep → 14 hours * 30 days = 420 hours.
    • This means that it costs costs about $33 per day to live in the apartment, or $2.3 per hour.
  • Study → Work → Net Worth Gain
    • You studied 30 hours per week for 3 years (30 hours per week * 52 weeks * 3 years = 4680 hours) to land a job that increases your net worth $10.000 every year (on average), for 30 years. A total of $300,000 of networth gains.
    • This means that you will gain about $64 for each hour invested in studying, or $274 per day.
  • Clothing
    • You buy pair of trousers for $50, and wear them twice per month throughout for 2 years (24 months * 2 times per month = 48 times these were used).
    • This means that it costs about $1 to wear them each time.
      • By contrast, if these trousers were only used 7 times, it would cost about $7.14 to wear them each time.
  • Gaming
    • You buy a Playstation 5 for $499.99 and 10 games ($60 each) totalling $1.100. You play 60 hours per year during 5 years (60 hours * 5 years = 300 hours).
    • This means that each playing hour costs about $3.6.
  • Phone
    • You buy an iPhone 15 Pro for $999, and use it 4.8 hours per day during 2 years (4.8 hours * 365 days * 2 years = 3504 hours).
    • This means that it costs $0.29 per hour of active usage.
  • Computer
    • You buy a MacBook Pro 13” for $$1299, and use it 3 times per week for 2.5 hours per session, throughout 7 years (3 times per week * 2.5 hours * 52 weeks per year * 7 years = 2730 hours)
    • This means that it costs $0.48 per hour of usage.
  • Musical Instrument
    • You buy a guitar for $100. You play the guitar once per week, for one hour, during 2 years. 52 weeks * 1 hour * 2 years = 104 hours.
    • This means that each hour playing the guitar has a cost of $1.
  • Video Streaming
    • You have a standard Netflix subscription that costs $15.99 per month, and watch 3.2 hours (3.2 hours * 30 days = 96 hours per month) of videos throughout that month.
    • This mean that each hour costs about $0.17.
  • Audio Streaming
    • You have a premium Spotify subscription that costs $10.99 per month, and listen to about 25 hours of music and podcasts throughout that month.
    • This means that each hour costs about $0.44.

We could go on with further examples, as this simple heuristic could be applied to several scenarios, and the cost cost benefit analysis could have any perspective one so desires, but I would invite you to visit your storage and search for something you acquired long ago that got used less times than expected, and do a rough calculation of what was it’s relative cost / benefit. The same for any other investment you did in the past.

Was it a surprising result?

How did it stack up against your initial expectations?


  1. This gross simplification is meant to keep the example simple, and does not take into account other direct and indirect costs, such as the computer needed to edit the videos, electricity costs, office space costs, labor costs. It also does not take into account the gained equity, this is, you could sell the equipment and get some of that investment back. This was not considered because if the equipment breaks its value is rendered to zero, and when / if it is sold, it would be for a fraction at the initial value, not only because its inherent value would likely decrease through the years, but also because of inflation.↩

]]> 2023-10-21T00:00:00+01:00 Importance of Goal Clarity for Delegation http://lopespm.github.io/notes/2023/09/30/goal_clarity_in_delegation.html http://lopespm.github.io/notes/2023/09/30/goal_clarity_in_delegation.html Summary: 在文章中，作者提到了75/25规则，即如果你对某个项目/任务的执行者正确执行的可能性约为75%，那么就可以进行委托。剩下的25%空间是为了让该人学习、探索、犯错误、创新和成长。 如果决定委托项目/任务，接下来会发生什么呢？ 目标清晰 作者在委托工作时面临的一个常见挑战是，在交付工作后，结果往往是不可预测和无法控制的。这种非结构化的方法经常让我思考如何通过减少委托范围来减少未完成风险，并且给被委派者带来了困惑，不知道他们应该如何进行。 来自《Who not How》一书的一点信息提供了解决这个问题的指导和结构： (…) 研究发现，团队在具有高度自治但目标不明确以及缺乏绩效反馈的情况下，表现比低度自治的团队要差。 据上述研究称，具有高度自治、目标明确和定期反馈结果的团队的表现会飙升。 除了定期反馈外，目标清晰非常重要，尽管听起来很明显，但在委托过程中可能会被忽视和淹没。 实践中的明确性 在意识到上述问题后，对于每个委托的工作，我把明确预期结果、服务完成的定义、重要的里程碑和截止日期、什么时候应该向我汇报问题以及每个参与者的责任作为首要任务。 所有上述内容都以明确而简洁的方式进行沟通，并且最好以公开文件形式进行记录，以增加问责性和减少歧义（根据情况，仅口头沟通这些可能已经足够，特别是对于简单的任务）。 委派公式 基于以上情况，委派公式如下： 委派 = [75/25规则 -> 目标明确 + 自主权 + 定期反馈]

Previously, in “Learnings as a Software Engineer Techlead”, I’ve alluded to the 75/25 rule, where if you are ~75% certain that someone will correctly execute a given project/task, then delegate. The other ~25% is the unknown space that the person needs in order to learn, explore, make errors, innovate and grow.

If the project/task was decided to be delegated, what comes next?

Clarity of Goals

One common challenge I’ve confronted when delegating was how unpredictable and uncontrollable the results felt upon handing over the work. This unstructured approach often left me thinking how I could decrease the non-completion risk by diminishing the delegated scope, and created confusion on the delegatee side on how they could proceed.

One tidbit of information from the “Who not How” book provided the guidance and structure to unlock this issue:

(…) research has found that teams who have high levels of autonomy but low clarity goals, as well as little performance feedback, perform worse than teams with low autonomy.

― Who not How, by Dan Sullivan with Dr. Benjamin Hardy

It is claimed in the above research that teams which have high autonomy, high goal clarity and regular feedback on their results, were observed to have their performance shooting through the roof.

Apart from regular feedback, goal clarity is incredibly important, and even though it might sound obvious, it can be overlooked and drowned out on the process of delegation.

Clarity, in Practice

After the above realization, I’ve made it a priority for each piece of delegated work to clearly define what was expected as an outcome, its definition of done, important milestones and deadlines, when should an issue be scaled to me, and which are the ownerships for each of the participants.

All of the above communicated in a clear and concise way, and preferably publicly documented to increase accountability and decrease ambiguity (depending on the situation, merely communicating these verbally might be enough, especially if the task is simple).

The Delegation Formula

Given the above, the proposed formula for delegation is:

Delegation = [ 75/25 rule -> Goal Clarity + Autonomy + Regular Feedback ]

]]> Life has several exits http://lopespm.github.io/notes/2023/08/19/life_exits.html http://lopespm.github.io/notes/2023/08/19/life_exits.html Life has several exits:

  • The exit from your workplace. The last day of work for your employer or your company.
  • The exit from that youth group that accompanied you through teenage years and early adulthood.
  • The exit from your high-school.
  • The exit from your hometown.
  • The exit from a long term relationship.
  • The exit from life. Your funeral.

Upon your exit, your contributions, actions, decisions, opinions and presence most likely will have affected the trajectories of the people and the world around you, in obvious and less obvious ways. Perhaps your own trajectory was deeply changed as well.

These exits may be celebrated, be a special day with fulfilling words and thoughts. They may be the last time you see a set of people. The last time you will see them in your entire life.

Perhaps upon your final exit, you have made such consequential dents on the world that your name will be engraved into the annals of history for centuries to come. On the other hand, contemporary personalities such Steve Jobs, Bill Gates, Elon Musk and Warren Buffet might be mostly forgotten a few centuries from now. People will move on, so most likely you should be grateful if your loved ones keep you in their thoughts for those ensuing decades.

Life has several exits. Life is also short and limited. There’ll be a finite number of exits you will experience. Make the most out of each period they enclose.

]]> The real question behind 'what do you want?' http://lopespm.github.io/notes/2023/08/05/real_question_behind_what_you_want.html http://lopespm.github.io/notes/2023/08/05/real_question_behind_what_you_want.html It’s easy to want something. If you are fortunate enough to be on income level 4 living in a peaceful nation, the world has so much to offer, within easy reach. Yet, the currencies required to enjoy them are limited: time, energy, health, money. Even though it is easy to make lists of everything one wants (typically occurring on yearly passages, such as new year’s and anniversaries), only a handful of them are achievable.

Smaller lists require sacrifices

Given that lengthy lists are unachievable, they need to be slimmed down to a manageable and realistic size. They need to be prioritized. Prioritization is an euphemism for sacrificing.

It’s easy to want something, it’s hard to choose it against another desirable one.

An example

At the start of each year, I produce a small list of 3 to 4 high priority goals and 1 optional low priority goal of what I want to achieve by end of it, along with a small description on why I want them, and their impact. This takes several hours, sometimes days to put together, because of all the other ones that need to be left out.

Once the list is made though, it’s incredibly powerful due to it’s easy recall. This means that whenever I am faced with conflicting preferences, I can easily remember it and know in a snap what is the right thing to do. That is the real power behind that list. It’s guiding all the tiny small steps and decisions happening throughout the year, bringing me progressivily closer to those goals.

The question

The real question behind defining knowing what we want is actually what we are willing to sacrifice. Once the latter is defined and we are 100% convinced about it, then knowing what we want is bullet proof and easy to follow through.

What are you willing to sacrifice?

]]> Writing as a form of thinking http://lopespm.github.io/notes/2023/07/02/writing-as-a-form-of-thinking.html http://lopespm.github.io/notes/2023/07/02/writing-as-a-form-of-thinking.html Large language models (LLM) like ChatGPT triggered a remarkable societal and computational shift that feels comparable to the impact from the initial internet era with its first impressive search engines like Google. LLMs are the toast of the town.

Writing emails, reports, school homework, analysis, code, summaries. The list goes on.

Save from when you would delegate this task to another fellow human, long hours would be passed in the past trying to conjure a piece of text that could communicate something to another entity. Now this can be made at scale, at low cost, with low effort.

Since we are still liable for the results of this tool, i.e. it’s still our name on the email sender, instead of just sending that email, we might first read that LLM output, interpret it, understand it, and then send it.

More than just text

Text is a form of communication. If something, or someone wrote it for us, certain decisions were made along the way to convey the goal that we gave. Out of the many paths possible to crystallize that piece of knowledge into a piece text, one of them was chosen.

I would claim that something gets lost on that delegation.

The writing process is more than just the production of text. Many times it requires the exploration of different perspectives, thinking deeply and coming to terms that we don’t know enough about a subject and need to learn more about it.

For example, it’s essential for me to have a notebook at hand to take notes during meetings and formal discussions. I write phrases, loose words, make small diagrams, jot down some reminders. Some of them are never to be re-read again, others I revisit to structure them down into a concise structure. Most of all, they help me think about a problem.

Same holds for notes and articles. I start with a cloud of loosely related ideas, which I attempt to refine into a structured form. Similar to the double diamond process.

Same for books. Several times I’ve come to terms that I learned close to nothing about a book read one month before. Or conversations. Or movies. Or experiences.

This is, except if I reflected or acted about them. Except if I wrote down my conclusions about them.

For me, taking notes helps make sure that I’m really thinking hard about what’s in there. If I disagree with the book, sometimes it takes a long time to read the books because I’m writing so much in the margin

― Bill Gates

]]> Custom Pomodoro Technique http://lopespm.github.io/notes/2023/07/02/custom-pomodoro-technique.html http://lopespm.github.io/notes/2023/07/02/custom-pomodoro-technique.html

Time is the most valuable thing a man can spend.

― Theophrastus

There are countless articles, books, videos and lectures about time management. Since all of us have different needs, goals and environments, it is best to face them as guidances, where we take the ones we need, and leave the others in our toolbelt for another time.

One of these methodologies is the Pomodoro Technique, in which a kitchen timer is used to break work into intervals, typically 25 minutes in length, separated by short breaks.

This doesn’t work for me, so here is my Custom Pomodoro Technique.

How it works

I usually need long bouts of deep focus during my work or any complex tasks. This can be reading / writing code, aligning with partners, structuring plans, going through my finances, etc. 25 minutes is typically not enough, since it coincides with where most of my flow peaks happen.

One hour is my target for continuous focused work. Below that, I know that I can still take advantage of the remaining time to productively go through a problem. Above that, I start to prepare to wind down and take a break.

Rule of thumb for breaks are to be enough to feel rested, which tends to happen in less than 15 minutes.

Tools

I’ve tried several tools to track time. Physical and digital. Complex and simple. One has stuck for several years, and is the only one I use right now: Klokki

Klokki is a concise Mac application that tracks time and generates nice time reports. It also allows you to track your time automatically, but I don’t use that feature.

This is a wonderful tool that fulfills my main requirements:

  • Being able to start and stop the timer with a single click.
  • The timer should be always visible, but also non-obtrusive.
  • Simple usage, with a low UI footprint. No distractions.
  • Concise time reports, which allow me to know how much time (and energy) I’ve used throughout the week. This is important for pacing calibration.

The rules

  1. Whenever I sit down at the computer and start working, I start the timer. I converged into using a single category to track all my deep flow blocks, since having multiple categories created too much mental overhead

  2. Whenever the timer is on, I’m in productivity / work mode, and no distractions outside my goals are allowed.

  3. Refrain from stopping the flow session before the one hour mark. Start preparing to stop the flow session after the 1 hour mark.

  4. Take breaks in-between flow sessions, until I feel rested. Normally less than 15 minutes.

  5. When the timer is off, scattered tasks, exploration and wandering are allowed and encouraged.

The benefits

This is an incredibly simple methodology that I’ve been using throughout the past years. The recurrent benefits I’ve captured from it were:

  • Clear boundaries between focus time and exploration.
  • Simple, easy to follow process, allowing for flexibility on how time blocks are managed. Easy to consistently abide by.
  • Pacing: prevents over-working or under-utilization of my energy / time.

Feel free to use this guidance / methodology and experiment with it, Or not 🙏

]]> Solving task switching through documentation http://lopespm.github.io/notes/2023/05/18/solving-task-switching-through-documentation.html http://lopespm.github.io/notes/2023/05/18/solving-task-switching-through-documentation.html We don’t actually multitask. We switch-task, rapidly shifting from one thing to another, interrupting ourselves unproductively, and losing time in the process 1. It just so happens that many aspects of our life require managing multiple tasks at the same time, so how to approach it?

One strategy that consistently worked for me has been using documentation to checkpoint each step of a process. Just like in a videogame where the last saved checkpoint can be recalled.

The Backtracking Inefficiency

Similar to a CPU, our short-term memory (STM) has access to a limited amount (around four or seven concepts have been suggested 2) of very quick memory. Everytime we change to a different task, most of the resident concepts need to be flushed in order to load the ones from the incoming task. This is quite taxing by itself and narrowly optimizable, since our wetware is mostly fixed. The biggest efficiency opportunities lie in where these concepts are loaded from.

When working on a task, often we grab several simultaneous concepts, combine them together and generate new ones, which are then added to the concept pool and built upon further. During this process we are faced with these options:

  • Option 1. Don’t store our working memory - essentially lose work. Let’s rule out this option.
  • Option 2. Progressively store the existing concepts into our long term memory.
  • Option 3. Progressively store the existing concepts into an external memory bank, such as a document or piece of paper.

Option 2. works well, but requires significant cognitive effort and often only keeps the most relevant conclusions. If for some reason these need to be re-considered or understood why they were generated in the first place, we backtrack into the seminal thought processes, which in turn will probably require a repetition of previous work.

This is one of the biggest offenders when task switching, because even if we recall the end state of a task (which is hard by itself), a lot of time and energy are spent when revisiting the overall context via backtracking.

Documentation as Checkpoint Mechanism

Enter Option 3.: Progressively store these existing concepts into an external memory bank, such as a document or piece of paper. Although writing in an external memory bank the contents of our working memory can take some effort, the yielded results greatly justify the initial investment:

  • Less cognitive load upon re-starting a task
  • Backtracking effort is reduced, since it is a matter of reading a document
  • Shareable with other stakeholders, which shaves off the time required to explain these concepts
  • Task switching becomes smoother and less burdensome

Parting Thoughts

  • The documentation’s structure significantly impacts how efficiently it can be parsed and shared. That is a whole new topic by itself.
  • Documentation as a checkpointing mechanism is a great way to consolidate personal learnings and reflections. It’s a valuable tool for leveraging life’s experiences to mature and grow. Option 1. is such a waste.

  1. Bregman, How (and Why) to Stop Multitasking↩

  2. About the Distinction between Working Memory and Short-Term Memory↩

]]> The case for not having a TV http://lopespm.github.io/notes/2023/05/04/the-case-for-not-having-tv.html http://lopespm.github.io/notes/2023/05/04/the-case-for-not-having-tv.html Listen to David Sinclair or Andrew Hubberman on how to extend lifespan or improve health markers, and a common trait emerges: the body strives and improves when exposed to just enough stress and struggle. Not too much, though. Cold baths, saunas, fasting, exercise, red light (which disrupts the way that our mitochondria make energy), eating stressed greens, are examples of just that.

On the other hand, if you spend your whole day sitting, or typing, always satisfying your hunger, your AMPK and mTOR pathways (which are master regulators of cell metabolism) and sirtuins say: “Hey, times are good. Let’s just grow tissue, go forth, multiply and not build a sustainable body in the long run.”

The idolized unhurdled, stable and comfortable life, the easy access to resources and high caloric food, can do more harm than good when left unattended.

The riveting third person

I’ve consumed gargantuous amounts of television throughout my youth, so I’ve grown familiar with it’s charming alure. In an almost frictionless experience, where it suffices to lean back and pick one of the channels or series/movies/documentaries readily suggested by a streaming provider, an experience is entered where carefully crafted packages and passionate stories are delivered to us, requiring only our attention. It can even serve as the third-person to liven up the room, when in a tedious moment with a loved one.

No real struggle, no real effort required. A passive experience. The body rejoices in it, like it does when ingesting a sugar packed sweet. Just like most, I enjoy that feeling. Just like most, I have a limited amount of willpower. Hence why I don’t have a TV at home. Neither do I have sweets.

I love it. I don’t love the fact that I’m blocking myself away from those satisfactions. No, I relish the long term results. The time it opens up to actively read, write (this post for example), create, think, be present with others, and even watch podcasts (of which I make an effort to pick an episode outside the normal recommendations). I’m not perfect, and the above serves as a rule for me. And rules do have exceptions, and that’s ok.

Still, this is the case for not having a TV.

]]> The First Note http://lopespm.github.io/notes/2023/04/07/first-note.html http://lopespm.github.io/notes/2023/04/07/first-note.html Welcome to Notes!

This is a new space where you will be able to find essays, notes and other quick thoughts, whereas the existing Blog section will remain allocated to denser article pieces.

Why?

The Recurrent Idea

I have a rule when buying non-essential items for my house: if I’ve been recurrently thinking about buying something over a span of multiple days, then it’s a good purchasing signal. The thought process for writing articles for this blog is similar.

The downside of this approach is that the ideas and thoughts that don’t make it to a blog article are relegated into the realm of personal writings and/or conversations with others, but never exposed to the public. On the other hand, this very same thirst to share quick thoughts was recurrent by itself.

Framing

The yearning towards developing this concept was there, but a satisfactory hosting plaform was missing still:

  • Twitter: although it allowed for volume and looser thoughts, the character restriction would limit the exploration of a topic, and all the content would not be owned by me.
  • New self-hosted space: this allowed me to own the contents and have a clear separation between dense blog articles and quick thoughts, but the maintenance cost was prohibitive.

The third option was to host them here, in Byte Tank, which was customized over several years and already had the workflow and components I strived for. But how to frame this new concept into the existing structure?

The answer came from Alin Panaitiu. On his website, Alin has a separate section for Notes, which makes perfect sense to me. Notes sets the expectation that they are not fully fledged articles, and that they cater towards volume, rather than density.

Making it a reality

An idea is only worth for its implementation, and these were the last nudges that incentivized me to bring it to life:

  • The inspiring examples presented in several of the comments in Ask HN: What has your personal website/blog done for you?. One of them being a reference to Why and how to write things on the Internet, the other one being Alin’s website (see above).
  • The spirit of the earlier Internet. I remember the earlier times of the Internet where there was this widespread fascination with the fact that with a telephone connection, one could have access to its immense world, and had the possibility to create their own virtual home, that anyone could visit. That is one of the many beauties of this massive network. Low cost, virtually no gatekeepers, nearly boundless. I find this idea incredibly exciting and empowering.
  • Even in an era where text content is now produced for a low cost and in massive amounts (GPT and other LLMs), I believe there is still space for artisanal human baked opinions. The human relatableness, its connection with the environment, biology and even its limitations are still charming on their own.
  • The incentive to create notes in volume could be a great way to generate more source material that can be coalesced into new blog articles
  • Exposing looser thoughts publicly forces me to structure then, and I believe this process can make me better equipped to navigate the complex universe of human interactions, in which having a more structured mindset can make a world of a difference. Personally and professionally.

Going Forward

This is an exciting new phase for this modest establishment, and I’m looking forward to populating this section with new content. Until then!

]]> Book Quotes Collection - Part 1 http://lopespm.github.io/2022/12/29/book-quotes-collection-part-one.html http://lopespm.github.io/2022/12/29/book-quotes-collection-part-one.html


Books are an immensely valuable source of condensed knowledge and insightful thoughts. Here is a selection of my favorite quotes, from books I’ve read throughout the past years:



Education

  • “No man can reveal to you aught but that which already lies half asleep in the dawning of your knowledge. 13
  • By the time a student gets to college, he’s spent a decade curating a bewilderingly diverse résumé to prepare for a completely unknowable future. Come what may, he’s ready—for nothing in particular. (…) what could be a more appropriate reward for two decades of résumé-building than a seemingly elite, process-oriented career that promises to “keep options open”? 1
  • The biggest battle I have is restricting their video game time because they want to play all the time. The rule is they have to read more than they play video games. They also can’t play completely stupid video games. There’s one game they downloaded recently called Cookies or something. You literally tap a fucking cookie. It’s like a Psych 101 experiment. I made them delete the cookie game. They had to play Flappy Golf instead, which is like Flappy 2

Work

  • But I say to you that when you work you fulfil a part of earth’s furthest dream, assigned to you when that dream was born, And in keeping yourself with labour you are in truth loving life, And to love life through labour is to be intimate with life’s inmost secret. 13
  • Promises are easy and cheap to make, actual work is hard and expensive. If it wasn’t, you’d just have done it now rather than promised it later. 11

Business / Companies

  • Startups Are Easy, Stayups Are Hard 11
  • On describes the earliest startup as like driving a race car. You’re close to the ground, and you feel every move you make. You have control, you can turn quickly, you feel like things are moving fast. Of course, you’re also at risk of crashing at any moment, but you only take yourself down if you do. As you grow, you graduate to a commercial flight. You’re farther from the ground, and more people’s lives depend on you, so you need to consider your movements more carefully, but you still feel in control and can turn the plane relatively quickly. Finally, you graduate to a spaceship, where you can’t make quick moves and the course is set long in advance, but you’re capable of going very far and taking tons of people along for the ride. 16
  • Most of the people at PayPal don’t understand this. The reason it worked was because the cost of transactions in PayPal was lower than any other system. And the reason the cost of transactions was lower is because we were able to do an increasing percentage of our transactions as ACH, or automated clearinghouse, electronic transactions, and most importantly, internal transactions. Internal transactions were essentially fraud-free and cost us nothing. An ACH transaction costs, I don’t know, like twenty cents or something. But it was slow, so that was the bad thing. It’s dependent on the bank’s batch processing time. And then the credit card transaction was fast, but expensive in terms of the credit card processing fees and very prone to fraud. That’s the problem Square is having now. “Square is doing the wrong version of PayPal. The critical thing is to achieve internal transactions. 2
  • Is this thing any good? Does it solve a real problem? Should we have made it better? Are we making what customers want? Is anybody going to buy this? Did we price it right? 11
  • There’s only a bunch of business-axiom baloney like “If you’re not growing, you’re dying.” Says who? 11
  • Tolstoy opens Anna Karenina by observing: “All happy families are alike; each unhappy family is unhappy in its own way.” Business is the opposite. All happy companies are different: each one earns a monopoly by solving a unique problem. All failed companies are the same: they failed to escape competition. 1
  • Are production incidents happening that are taking up lots of time? Are a bunch of people sick? Are they bickering over coding style in their code review comments? Are the tickets that are being written vague, too big, too small? Does the team seem upbeat in their communication style, sharing fun things as well as important work in chat, or are they purely business? Look at their calendars. Is the team spending many hours a week in meetings? Is their manager not doing 1-1s? None of these things are necessarily smoking guns, but they may point to an area to address. 16
  • Conway’s Law is often cited in discussions of this kind of structure. It states: “Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” 16
  • As ridiculous and dangerous as this might seem, plenty of teams fall prey to the lure of status. These often include altruistic nonprofit organizations that come to believe that the nobility of their mission is enough to justify their satisfaction. Political groups, academic departments, and prestigious companies are also susceptible to this dysfunction, as they often see success in merely being associated with their special organizations. 17

Planning

  • Forget “minimum viable products”—ever since he started Apple in 1976, Jobs saw that you can change the world through careful planning, not by listening to focus group feedback or copying others’ successes. 1
  • Because let’s face it: Goals are fake. Nearly all of them are artificial targets set for the sake of setting targets. 11
  • Start with a shared understanding of the goals, risks, and the questions to answer before making a decision. When you assign the ownership for making a 16
  • “We who cut mere stones must always be envisioning cathedrals.” 9
  • You have 10 productive engineering weeks per engineer per quarter 16
  • Dedicate 20% of your team’s schedule to “sustaining engineering.” 16
  • It’s likely that Q1 (immediately after the winter holidays) will be the most productive and Q4 (the quarter that includes winter and the end-of-year holidays) will be the least productive. 16
  • Therefore, you must always be aggressive about sharing estimates and updates to estimates, even when people don’t ask, especially if you believe that the project is critical or likely to take longer than a few weeks. This means you must be aggressive about getting estimates, and as we all know, software estimation is a very difficult process. Negotiating the process that your team uses to estimate, on what timescale, for what projects, may be part of your job at this level. 16
  • If you haven’t taken the time to articulate the value of this work, it will get pushed aside in favor of projects that are more clearly valuable. 16
  • My advice is to do your best to gather data to support yourself, and talk about what will be possible when the work is done. 16
  • When an engineer comes to you with an engineering project that she wants to do, think about framing the project by answering these questions: How big is that project? How important is it? Can you articulate the value of that project to anyone who asks? What would successful completion of the project mean for the team? 16
  • The hardest part, in many ways, was getting started. The second-hardest part was getting comfortable making a guess about the future with highly imperfect information. 16
  • “The key, of course, is to define our goals, our results, in a way that is simple enough to grasp easily, and specific enough to be actionable. Profit is not actionable enough. It needs to be more closely related to what we do on a daily basis. (…) By combining some and eliminating others, they narrowed them to seven: revenue, expenses, new customer acquisition, current customer satisfaction, employee retention, market awareness, and product quality. 17

Management

  • The most important lesson I’ve learned is that you have to be able to manage yourself if you want to be good at managing others. The more time you spend understanding yourself, the way you react, the things that inspire you, and the things that drive you crazy, the better off you will be. 16
  • Contrary to the notion that teams waste time and energy arguing, those that avoid conflict actually doom themselves to revisiting issues again and again without resolution. They often ask team members to take their issues “off-line,” which seems to be a euphemism for avoiding dealing with an important topic, only to have it raised again at the next meeting. 17
  • Great teams ensure that everyone’s ideas are genuinely considered, which then creates a willingness to rally around whatever decision is ultimately made by the group. And when that is not possible due to an impasse, the leader of the team is allowed to make the call. 17
  • (…) dysfunctional teams that try to hedge their bets and delay important decisions until they have enough data to feel certain that they are making the right decision. As prudent as this might seem, it is dangerous because of the paralysis and lack of confidence it breeds within a team. 17
  • More than any other member of the team, the leader must be comfortable with the prospect of making a decision that ultimately turns out to be wrong. And the leader must be constantly pushing the group for closure around issues, as well as adherence to schedules that the team has set. What the leader cannot do is place too high a premium on certainty or consensus. 17
  • (…) the most effective and efficient means of maintaining high standards of performance on a team is peer pressure. One of the benefits is the reduction of the need for excessive bureaucracy around performance management and corrective action. More than any policy or system, there is nothing like the fear of letting down respected teammates that motivates people to improve their performance. 17
  • If the boss really wants to know what’s going on, the answer is embarrassingly obvious: They have to ask! Not vague, self-congratulatory bullshit questions like “What can we do even better?” but the hard ones like “What’s something nobody dares to talk about?” or “Are you afraid of anything at work?” or “Is there anything you worked on recently that you wish you could do over?” Or even more specific ones like “What do you think we could have done differently to help Jane succeed?” or “What advice would you give before we start on the big website redesign project?” 11
  • A lack of engagement in meetings tends to mean the team isn’t engaged by the work or do not feel like they have a say in the decision-making process. 16
  • But there is such a thing as artificial harmony, and conflict-avoidant managers tend to favor harmony above functional working relationships. 16
  • The goal is to identify problems that are causing the team to work less effectively together and resolve them, not to become the team’s therapist. 16
  • Many of us believe that the way to be liked is to be seen as nice — that niceness is itself the goal. Your goal as a manager, however, should not be to be nice, it should be to be kind. 16
  • It’s kind to tell someone who isn’t ready for promotion that she isn’t ready, and back that up with the work she needs to do to get there. It’s unkind to lead that person on, saying “Maybe you could get promoted,” and then watch her fail. It’s kind to tell someone that his behavior in meetings is disrupting the group. It’s awkward, and uncomfortable, but it’s also part of your job as his manager to have these difficult conversations. 16
  • Thinking about your actions is the best way to combat fear of conflict. Am I pushing this decision to the team because they really are the best people to decide, or am I just afraid that if I make an unpopular but necessary decision people will be mad at me? Am I avoiding working through this issue with my peer because she’s truly difficult to work with, or am I just hoping that the issue will resolve itself because I don’t want to have to discuss it and possibly be wrong? Am I holding back on giving my employee this feedback because he really was having a bad day and it’s just a one-off, or am I holding back because I’m afraid he won’t like me as a manager if I tell him? Be thoughtful about your behavior, and it’s unlikely that you’ll seek out unnecessary conflict. 16
  • The real goal here is psychological safety — that is, a team whose members are willing to take risks and make mistakes in front of one another. 16
  • Your first goal is to protect your team as a whole, the second is to protect each individual on the team, and your last priority is protecting yourself. 16
  • Simply put, if your team member doesn’t respect you or her peers, why is she working there? 16
  • So at the end of another day when I feel like I didn’t write enough code and I have no way to quantify what I’ve achieved, I tell myself I was being as good a manager as I know how to be. 16
  • Project management. Onboarding new team members. Working with the product team to break down product roadmap goals into technical deliverables. Production support. These are all skills members of your team need to learn. 16
  • Even when you have been hired to fix a team, remember that the company has gotten this far because of some fundamental strengths. 16
  • Whether it’s QA, design, product management, or technical operations, ask lots of questions, but in an open way. Make it clear to the person that your goal is to understand what she does so that you’re capable of appreciating it better. 16
  • Sit in their meetings. Are they boring to you? Is the team bored? Who is speaking most of the time? Are there regular meetings with the whole team where the vast majority of the time is spent listening to the manager or product lead talk? Boring meetings are a sign. They may be a sign of inefficient planning on the part of the organizers. There may be too many meetings happening for the information covered. They may indicate that the team members don’t feel they can actually help set the direction of the team, or choose the work that will happen. They often signal a lack of healthy conflict on a team. Good meetings have a heavy discussion element, where opinions and ideas are drawn out of the team. If the meetings are overscripted, so that no real conversation can take place, it stifles that creative discussion. If people are afraid to disagree or bring up issues for fear of dealing with conflict, or if managers always shut down conflict without letting disagreements air, this is a sign of an unhealthy team culture. 16
  • A bunch of people who never talk to each other and are always working on independent projects are not really working as a team. There would be nothing wrong with that if the team were performing well, but given that they’re not, this may be contributing to your problem. 16
  • Projects change. Teams may even be disbanded or moved around in ways that you don’t understand or agree with. As a manager, the best thing you can do is help people feel capable of tying up loose ends, stabilizing the current in-flight projects, and easing into their new work in a controlled fashion. This is an area where you can and should push back. Make sure that your teams get adequate time to finish up current work. Furthermore, push for engineering involvement in the early planning for the new work so that people can get excited about the projects they are moving on to. 16
  • It’s not about being the lead engineer, chasing the latest language or framework, or having the shiniest technology. It’s about building a team with the tools and attributes to build the best possible product for our customers. 16
  • You can’t give up the responsibility of management without giving up the power that comes with it. 16
  • Reach through the part of you that is shy about praising people or embarrassed to share your feelings, and go into the part of you that cares about the people you work with. You can share these stories in a way that is not forced or fake. 16
  • (…) the larger the company, the smaller the team should be at the top. And with the addition of a new head of sales and a human resources director, her staff had grown to a barely manageable eight. It wasn’t that Kathryn couldn’t handle the weekly one-on-ones, but it was increasingly difficult to have fluid and substantive discussions during staff meetings with nine people sitting around the table. 17
  • (…) trust is the confidence among team members that their peers’ intentions are good, and that there is no reason to be protective or careful around the group. (…) Teams that lack trust waste inordinate amounts of time and energy managing their behaviors and interactions within the group. They tend to dread team meetings, and are reluctant to take risks in asking for or offering assistance to others. As a result, morale on distrusting teams is usually quite low, and unwanted turnover is high. 17

Culture

  • Initially the technologists and the hippies did not interface well. Many in the counterculture saw computers as ominous and Orwellian, the province of the Pentagon and the power structure. In The Myth of the Machine, the historian Lewis Mumford warned that computers were sucking away our freedom and destroying “life-enhancing values.” An injunction on punch cards of the period—“Do not fold, spindle or mutilate”—became an ironic phrase of the antiwar Left. 3
  • Engineers frequently struggle with the transition to respecting and communicating well with diverse peers. I believe the struggle with respect is a side effect of the current tech culture, which tells us that engineers are the smartest people in the room. It can’t be said strongly enough: your peers who are not analytically driven are not stupid. 16
  • “Engineers who graduated from MIT” is not a culture. “People who value technology innovation, hard work, intellect, scientific process, and data” might be. 16
  • A lot of interviews try to determine cultural fit by what I would call “friendship” markers, such as “Would you like being stuck in an airport with this person?” 16
  • Furthermore, culture fit as determined by friendship tests is almost certain to be discriminatory in some way. Humans form friendships with people who have significant shared background experiences, and these experiences tend to closely correlate with things like schooling, race, class, and gender. The shortcuts you get by hiring friends are not usually the values you need to form a strong team. 16

Systems / Processes

  • A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. 16
  • As your teams and systems grow, it’s almost impossible for any one person to keep the systems in her head. Because we have a bunch of people coordinating work, we evolve processes around that work coordination in order to make risks obvious. 16
  • (…) you should not put a complicated process on any activity where you want people to move quickly and where you believe the risk for change in that activity is low or that the risks themselves are obvious to the whole team. If you want to do code review for all changes, make sure that the process for code review is not so onerous that the team slows down significantly on minor changes, because that will impact your whole group’s productivity. The second implication is that you need to be on the lookout for places where there is hidden risk, and draw those hidden risks out into the open. There’s a saying in politics that “a good political idea is one that works well in half-baked form,” and the same goes for engineering processes. The processes should have value even when they are not followed perfectly, and that value should largely lie in the act of socializing change or risk to the team as a whole. 16
  • Be specific about the kinds of changes that need architecture review. Usually these include new languages, new frameworks, new storage systems, and new developer tooling. 16

Focus on what is important

  • But in effect it was unanswerable even now, since the few scattered survivors from the ancient world were incapable of comparing one age with another. They remembered a million useless things, a quarrel with a workmate, a hunt for a lost bicycle pump, the expression on a long-dead sister’s face, the swirls of dust on a windy morning seventy years ago: but all the relevant facts were outside the range of their vision. They were like the ant, which can see small objects but not large ones. 7
  • It’s lonely at the top. Ninety-nine percent of people in the world are convinced they are incapable of achieving great things, so they aim for the mediocre. The level of competition is thus fiercest for “realistic” goals, paradoxically making them the most time and energy-consuming. It is easier to raise \$1,000,000 than it is \$100,000. It is easier to pick up the one perfect 10 in the bar than the five 8s. If you are insecure, guess what? The rest of the world is, too. Do not overestimate the competition and underestimate yourself. You are better than you think. 8
  • One does not accumulate but eliminate. It is not daily increase but daily decrease. The height of cultivation always runs to simplicity. 8
  • “Five years of reporting on attention have confirmed some home truths,” Gallagher reports. “[Among them is the notion that] ‘the idle mind is the devil’s workshop’ when you lose focus, your mind tends to fix on what could be wrong with your life instead of what’s right.” 9
  • You have a finite amount of willpower that becomes depleted as you use it. 9
  • Urgency is often more clearly felt than importance. 16
  • “If everything is important, then nothing is.” 17

Choices / Decisions

  • “Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to,” said the Cat. “I don’t much care where —” said Alice. “Then it doesn’t matter which way you go,” said the Cat (quoted from Lewis Carroll, Alice in Wonderland) 8
  • (…) a decision is better than no decision. They also realize that it is better to make a decision boldly and be wrong—and then change direction with equal boldness—than it is to waffle. 17
  • If you’re 25 and announce you want to start a new circus, the response is different: Be realistic; become a lawyer or an accountant or a doctor, have babies, and raise them to repeat the cycle. 8
  • It was curious how that predestined horror moved in and out of one’s consciousness. There it lay, fixed in future times, preceding death as surely as 99 precedes 100. One could not avoid it, but one could perhaps postpone it: and yet instead, every now and again, by a conscious, wilful act, one chose to shorten the interval before it happened. 7
  • Much of your pain is self-chosen. 13
  • No is easier to do, yes is easier to say. No is no to one thing. Yes is no to a thousand things. (…) No is calm but hard. Yes is easy but a flurry. Knowing what you’ll say no to is better than knowing what you’ll say yes to. 11
  • (…) so practice getting comfortable with the quick no (and the quick yes!) for low-risk, low-impact decisions. 16
  • When you want to allow for group decision making, the group needs to have a clear set of standards that they use to evaluate decisions. Start with a shared understanding of the goals, risks, and the questions to answer before making a decision. When you assign the ownership for making a decision to someone on the team, make it clear which members of the team should be consulted for feedback and who needs to be informed of the decision or plan. 16
  • Taking conflicting perspectives and incomplete information and setting a direction, knowing that the consequences of a poor decision will impact both you and possibly the whole team. If making decisions were easy, there would be much less need for managers and leaders. However, as anyone who has spent a lot of time managing can tell you, making decisions is one of the most draining and stressful parts of the job. 16

Get Stuff Done / Productivity

  • People are fond of using the “it’s not what you know, it’s who you know” adage as an excuse for inaction, as if all successful people are born with powerful friends. 8
  • Tomorrow becomes never. No matter how small the task, take the first step now! 8
  • It’s not uncommon for people to pick up their phones dozens of times a day when some push notification makes it buzz, because WHAT IF IT WAS SOMETHING SUPER IMPORTANT! (It just about never is.) 11
  • “There is nothing so useless as doing efficiently that which should not be done at all.” 11
  • “I know some of my classmates pursued the alternative scattershot technique with some success, but that’s not my bag,” Marrinan said, explaining his perseverance. “I deal with rejection by persisting, not by taking my business elsewhere. My maxim comes from Samuel Beckett, a personal hero of mine: ‘Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.’ You won’t believe what you can accomplish by attempting the impossible with the courage to repeatedly fail better.” 8
  • Success is not a matter of mastering subtle, sophisticated theory, but rather of embracing common sense with uncommon levels of discipline and persistence. 17

Solving problems

  • She provides the example of a couple fighting over inequitable splitting of household chores. “Rather than continuing to focus on your partner’s selfishness and sloth,” she suggests, “you might focus on the fact that at least a festering conflict has been aired, which is the first step toward a solution to the problem, and to your improved mood.” 9
  • Your boss may not want to be stuck solving your problems, but you can bet that she’ll be happy to provide feedback if you phrase it as needing advice. 16

Accountabilty

  • “I’m using you as an example, Carlos, because you are an easy person to let off the hook. But this could apply to anyone. Some people are hard to hold accountable because they are so helpful. Others because they get defensive. Others because they are intimidating. I don’t think it’s easy to hold anyone accountable, not even your own kids.” 17
  • “You are fighting. But about issues. That’s your job. Otherwise, you leave it to your people to try to solve problems that they can’t solve. They want you to hash this stuff out so they can get clear direction from us.” 17
  • In fact, team members who are particularly close to one another sometimes hesitate to hold one another accountable precisely because they fear jeopardizing a valuable personal relationship. Ironically, this only causes the relationship to deteriorate as team members begin to resent one another for not living up to expectations and for allowing the standards of the group to erode. 17

Communication

  • In my experience, most people need to hear something at least three times before it really sinks in. You’re going to tell your own senior management and leadership team. You’ll hold an all hands meeting. You may need to send some email detailing the changes as well. A little bit of communication planning can go a long way in such situations. Try to anticipate the questions you might get and prepare answers for those questions. Be as clear as possible about the projects or structure to be changed, so there’s little room for confusion. And don’t forget to sell this change as a good thing! 16

Human condition

  • ‘When you make love you’re using up energy; and afterwards you feel happy and don’t give a damn for anything. They can’t bear you to feel like that. They want you to be bursting with energy all the time. All this marching up and down and cheering and waving flags is simply sex gone sour. If you’re happy inside yourself, why should you get excited about Big Brother and the Three-Year Plans and the Two Minutes Hate and all the rest of their bloody rot?’ 7
  • Having so recently been one of the underdogs of the savannah, we are full of fears and anxieties over our position, which makes us doubly cruel and dangerous. 5
  • I define masculinity as that part of a man that equips him to survive as an individual, clan, and species. Without this masculine energy we would have all become extinct eons ago. Masculinity empowers a man to create and produce. It also empowers him provide for and protect those who are important to him. These aspects of masculinity include strength, discipline, courage, passion, persistence, and integrity. 10
  • This frustration is due to the reality that in general, women view men who try to please them as weak and hold these men in contempt. Most women do not want a man who tries to please them — they want a man who knows how to please himself. Women consistently share with me that they don’t want a passive, pleasing wimp. They want a man — someone with his balls still intact 10
  • When recovering Nice Guys create relationships with people who don’t need fixing, they improve their odds of finding the love they want. This doesn’t mean searching for the perfect partner, just one who is already taking responsibility for her own life. 10
  • Internalized shame and fear are the greatest barriers to a satisfying sex life. A man can read all the books he wants on “How To Pick Up Women” or watch all the instructional videos on improving sexual technique. None of these things will help him get good sex as long as he has shame and fear about being sexual or being a sexual being. 10
  • “I am pretty afraid to fail. And so I tend to over-engineer things and do them myself. I don’t like to tell other people what to do, which, ironically, only makes it more likely that I’m going to fail.” For the slightest moment, Jeff seemed to fight back emotions and then recovered instantly. He was sure no one noticed. “And I think that’s probably the biggest reason that we haven’t succeeded, and that I’m not the CEO anymore.” He paused, and then added quickly, “Which I’m okay with, really. In fact, I’m pretty happy to be out of that job.” 17

Relationships

  • All great relationships, the ones that last over time, require productive conflict in order to grow. This is true in marriage, parenthood, friendship, and certainly business. 17
  • And when one of you falls down he falls for those behind him, a caution against the stumbling stone. Ay, and he falls for those ahead of him, who though faster and surer of foot, yet removed not the stumbling stone. 13
  • Your friend is your needs answered. (…) For that which you love most in him may be clearer in his absence, as the mountain to the climber is clearer from the plain. And let there be no purpose in friendship save the deepening of the spirit. (…) For it is his to fill your need, but not your emptiness. 13
  • Former couples who break up because they don’t see a future together can remain friends, and the same goes for former employees who simply need a different team or company to shine. 16
  • “Two households, both alike in dignity.” The two houses are alike, yet they hate each other. They grow even more similar as the feud escalates. Eventually, they lose sight of why they started fighting in the first place. 1
  • All the evidence in the world can’t change a person who doesn’t want to change. 16
  • A very common clash occurs between people who are extremely analytically driven and those who are more creatively or intuitively focused. Another is between the people who prefer to embrace agility and change (and, yes, sometimes disorder) and those who push for more long-term planning, deadlines, and budgets. 16
  • “Set me like a seal upon thy heart, love is as strong as death.” 12
  • For even as love crowns you so shall he crucify you. Even as he is for your growth so is he for your pruning. 13
  • But if in your fear you would seek only love’s peace and love’s pleasure, Then it is better for you that you cover your nakedness and pass out of love’s threshing-floor, Into the seasonless world where you shall laugh, but not all of your laughter, and weep, but not all of your tears. 13
  • Your children are not your children. (…) You are the bows from which your children as living arrows are sent forth. 13

Behaviour

  • She knew the whole drivelling song by heart, it seemed. Her voice floated upward with the sweet summer air, very tuneful, charged with a sort of happy melancholy. One had the feeling that she would have been perfectly content, if the June evening had been endless and the supply of clothes inexhaustible, to remain there for a thousand years, pegging out diapers and singing rubbish. It struck him as a curious fact that he had never heard a member of the Party singing alone and spontaneously. It would even have seemed slightly unorthodox, a dangerous eccentricity, like talking to oneself. Perhaps it was only when people were somewhere near the starvation level that they had anything to sing about. 7
  • As recovering Nice Guys begin to surrender, dwell in reality, express their feelings, face their fears, develop integrity, and set boundaries, they access a power that allows them to welcome and embrace the challenges and “gifts” of life. Life isn’t a merry-go-round, it’s a roller coaster. As they reclaim personal power, recovering Nice Guys can experience the world in all of its serendipitous beauty. Life won’t always be smooth, it may not always be pretty, but it will be an adventure — one not to be missed. 10
  • As Nice Guys try to avoid the dark side of their masculinity, they also repress many other aspects of this male energy force. As a result, they often lose their sexual assertiveness, competitiveness, creativity, ego, thirst for experience, boisterousness, exhibitionism, and power. 10
  • (Activities to break free from the Nice Guy Syndrome) 10
    • If it frightens you, do it.
    • Don’t settle. Every time you settle, you get exactly what you settled for.
    • Put yourself first.
    • No matter what happens, you will handle it.
    • Whatever you do, do it 100%.
    • If you do what you have always done, you will get what you have always got.
    • You are the only person on this planet responsible for your needs, wants, and happiness.
    • Ask for what you want.
    • If what you are doing isn’t working, try something different.
    • Be clear and direct.
    • Learn to say “no.”
    • Don’t make excuses.
    • If you are an adult, you are old enough to make your own rules.
    • Let people help you.
    • Be honest with yourself.
    • Do not let anyone treat you badly. No one. Ever.
    • Remove yourself from a bad situation instead of waiting for the situation to change.
    • Don’t tolerate the intolerable — ever.
    • Stop blaming. Victims never succeed.
    • Live with integrity. Decide what feels right to you, then do it.
    • Accept the consequences of your actions.
    • Be good to yourself.
    • Think “abundance.”
    • Face difficult situations and conflict head on.
    • Don’t do anything in secret.
    • Do it now.
    • Be willing to let go of what you have so you can get what you want.
    • Have fun. If you are not having fun, something is wrong.
    • Give yourself room to fail. There are no mistakes, only learning experiences.
    • Control is an illusion. Let go; let life happen.
  • For to be idle is to become a stranger unto the seasons, and to step out of life’s procession, that marches in majesty and proud submission towards the infinite. 13
  • Have you peace, the quiet urge that reveals your power? Have you remembrances, the glimmering arches that span the summits of the mind? Have you beauty, that leads the heart from things fashioned of wood and stone to the holy mountain? Tell me, have you these in your houses? Or have you only comfort, and the lust for comfort, that stealthy thing that enters the house a guest, and then becomes a host, and then a master? 13
  • (about crime and punishment) And when the black thread breaks, the weaver shall look into the whole cloth, and he shall examine the loom also. 13
  • If either your sails or your rudder be broken, you can but toss and drift, or else be held at a standstill in mid-seas. For reason, ruling alone, is a force confining; and passion, unattended, is a flame that burns to its own destruction. (…) And since you are a breath in God’s sphere, and a leaf in God’s forest, you too should rest in reason and move in passion. ​​13
  • And the longer it lasted the more deadly it became. “It is as if I had been going downhill while I imagined I was going up. And that is really what it was. I was going up in public opinion, but to the same extent life was ebbing away from me. And now it is all done and there is only death. 14
  • Above all, don’t lie to yourself. The man who lies to himself and listens to his own lie comes to such a pass that he cannot distinguish the truth within him, or around him, and so loses all respect for himself and for others. 15
  • There is only one means of salvation, then take yourself and make yourself responsible for all men’s sins, that is the truth, you know, friends, for as soon as you sincerely make yourself responsible for everything and for all men, you will see at once that it is really so, and that you are to blame for every one and for all things. But throwing your own indolence and impotence on others you will end by sharing the pride of Satan and murmuring against God. 15
  • And people have even ceased to feel the impulse to self‐criticism. Don’t be like everyone else, even if you are the only one.” 15
  • Don’t be afraid. Conflict avoidance often arises from fear. We’re scared of the responsibility of making the decision. We’re afraid of seeming too demanding. We’re afraid people will quit if we give them uncomfortable feedback. We’re afraid people won’t like us, or that we’ll fail when we take this risk. Some fear is natural, and being sensitive to the outcomes of conflict is a wise habit. 16
  • Conflict avoidance runs deep in most of us, and matters of what feels like personal style can be particularly difficult. 16
  • “I don’t think anyone ever gets completely used to conflict. If it’s not a little uncomfortable, then it’s not real. The key is to keep doing it anyway.” 17
  • This background, combined with a strong desire to be seen as smart and “right,” sometimes led to some less-than-productive conversations about technology directions. I’d find myself arguing the merits of a language choice or technology purely based on the technical merits. When this happened I became another engineer arguing in a group of engineers. It took me a long time to realize that my job wasn’t to be the smartest person in the room. It wasn’t to be “right.” Rather, my role was to help the team make the best possible decisions and help them implement them in a sustainable and efficient way. 16
  • “Wanting to be a CTO (or VP of Engineering) is like wanting to be married. Remember that it’s not just the title, it’s also the company and the people that matter.” Titles are definitely not everything. 16
  • Throwing out jargon to people who aren’t familiar with it — and who don’t even need to be familiar with it — makes us look stupid to them. 16
  • “Okay, first of all, don’t ever slam one of your teammates when that person isn’t in the room. I don’t care what you think of Mikey. She is part of this team, and you have to take your issues to her directly, or to me. You’re going to have to make that right.” 17

Group Thinking / Living in Society

  • They were talking about the Lottery. Winston looked back when he had gone thirty metres. They were still arguing, with vivid, passionate faces. The Lottery, with its weekly pay-out of enormous prizes, was the one public event to which the proles paid serious attention. It was probable that there were some millions of proles for whom the Lottery was the principal if not the only reason for remaining alive. It was their delight, their folly, their anodyne, their intellectual stimulant. Where the Lottery was concerned, even people who could barely read and write seemed capable of intricate calculations and staggering feats of memory. There was a whole tribe of men who made a living simply by selling systems, forecasts, and lucky amulets. 7
  • Whenever you find yourself on the side of the majority, it is time to pause and reflect. 8
  • It takes a tribe to raise a human. 5
  • Sociological research has shown that the maximum ‘natural’ size of a group bonded by gossip is about 150 individuals. 5
  • (…) for the craving for universal unity is the third and last anguish of men. Mankind as a whole has always striven to organize a universal state. 15
  • “What do you mean by isolation?” I asked him. “Why, the isolation that prevails everywhere, above all in our age—it has not fully developed, it has not reached its limit yet. For every one strives to keep his individuality as apart as possible, wishes to secure the greatest possible fullness of life for himself; but meantime all his efforts result not in attaining fullness of life but self‐destruction, for instead of self‐realization he ends by arriving at complete solitude. All mankind in our age have split up into units, they all keep apart, each in his own groove; each one holds aloof, hides himself and hides what he has, from the rest, and he ends by being repelled by others and repelling them. He heaps up riches by himself and thinks, ‘How strong I am now and how secure,’ and in his madness he does not understand that the more he heaps up, the more he sinks into self‐destructive impotence. For he is accustomed to rely upon himself alone and to cut himself off from the whole; he has trained himself not to believe in the help of others, in men and in humanity, and only trembles for fear he should lose his money and the privileges that he has won for himself. Everywhere in these days men have, in their mockery, ceased to understand that the true security is to be found in social solidarity rather than in isolated individual effort. 15
  • “Madness is rare in individuals—but in groups, parties, nations, and ages it is the rule,” Nietzsche wrote (before he went mad). If you can identify a delusional popular belief, you can find what lies hidden behind it: the contrarian truth. 1

Thoughts / Mindset

  • “Shallow men believe in luck, believe in circumstances. Strong men believe in cause and effect.” 1
  • (…) none of these things exists outside the stories that people invent and tell one another. There are no gods in the universe, no nations, no money, no human rights, no laws, and no justice outside the common imagination of human beings. 5
  • “He who is abandoned is an abandoner,” 3
  • The invitation read, “There’s an old Hindu saying that goes, ‘In the first 30 years of your life, you make your habits. For the last 30 years of your life, your habits make you.’ Come help me celebrate mine.” 3
  • (…) and more importantly, in order to take full advantage of the how, you need to understand the why. 4
  • Will they reach the nursery in time? If so, how delightful for them, and we shall all breathe a sigh of relief, but there will be no story. On the other hand, if they are not in time, I solemnly promise that it will all come right in the end. 6
  • What are you putting off out of fear? Usually, what we most fear doing is what we most need to do. 8
  • The existential vacuum manifests itself mainly in a state of boredom. 8
  • There is a direct correlation between an increased sphere of comfort and getting what you want. 8
  • I define personal power as a state of mind in which a person is confident he can handle whatever may come. This kind of power not only successfully deals with problems, challenges and adversity, it actually welcomes them, meets them head on, and is thankful for them. Personal power isn’t the absence of fear. Even the most powerful people have fear. Personal power is the result of feeling fear, but not giving in to the fear. 10
  • “Comparison is the death of joy.” 11
  • Yet the timeless in you is aware of life’s timelessness, And knows that yesterday is but today’s memory and tomorrow is today’s dream. 13
  • Men will even give their lives if only the ordeal does not last long but is soon over, with all looking on and applauding as though on the stage. 15
  • (…) for all is like an ocean, all is flowing and blending; a touch in one place sets up movement at the other end of the earth. 15
  • (…) fools are made for wise men’s profit.” 15
  • “There’s no such thing as a well-adjusted public figure. If they were well adjusted they wouldn’t try to be a public figure.” 2
  • “Men love the downfall and disgrace of the righteous” 15
  • Goodness is one thing with me and another with a Chinaman, so it’s a relative thing. Or isn’t it? Is it not relative? A treacherous question! 15
  • (…) “happiness and positivity is a choice,” the teammate who pooh-poohs every idea and criticizes everything is going to have problems fitting in. Sometimes, people will change to adopt the values. 16
  • “Politics is when people choose their words and actions based on how they want others to react rather than based on what they really think.” 17
  • Ideological conflict is limited to concepts and ideas, and avoids personality-focused, mean-spirited attacks. 17
  • You talk when you cease to be at peace with your thoughts; And when you can no longer dwell in the solitude of your heart you live in your lips, and sound is a diversion and a pastime. And in much of your talking, thinking is half murdered. 13

Wealth

  • And what follows from this right of multiplication of desires? In the rich, isolation and spiritual suicide; in the poor, envy and murder; for they have been given rights, but have not been shown the means of satisfying their wants. They maintain that the world is getting more and more united, more and more bound together in brotherly community, as it overcomes distance and sets thoughts flying through the air. 15
  • (…) yet to the very end he persisted in hoping that he would get that three thousand, that the money would somehow come to him of itself, as though it might drop from heaven. That is just how it is with people who, like Dmitri, have never had anything to do with money, except to squander what has come to them by inheritance without any effort of their own, and have no notion how money is obtained. 15

History / Evolution

  • Just 6 million years ago, a single female ape had two daughters. One became the ancestor of all chimpanzees, the other is our own grandmother. 5
  • Women who gave birth earlier, when the infants brain and head were still relatively small and supple, fared better and lived to have more children. Natural selection consequently favoured earlier births. And, indeed, compared to other animals, humans are born prematurely, when many of their vital systems are still under-developed. A colt can trot shortly after birth; a kitten leaves its mother to forage on its own when it is just a few weeks old. Human babies are helpless, dependent for many years on their elders for sustenance, protection and education. 5
  • Some scholars believe there is a direct link between the advent of cooking, the shortening of the human intestinal track, and the growth of the human brain. 5
  • It is unsettling – and perhaps thrilling – to think that we Sapiens could at one time have sex with an animal from a different species, and produce children together. 5

Religion / Spirituality

  • “I think different religions are different doors to the same house. Sometimes I think the house exists, and sometimes I don’t. It’s the great mystery.” 3
  • For even those who have renounced Christianity and attack it, in their inmost being still follow the Christian ideal 15
  • Freedom, free thought and science, will lead them into such straits and will bring them face to face with such marvels and insoluble mysteries, that some of them, the fierce and rebellious, will destroy themselves, others, rebellious but weak, will destroy one another, while the rest, weak and unhappy, will crawl fawning to our feet and whine to us: “Yes, you were right, you alone possess His mystery, and we come back to you, save us from ourselves!” 15

Geopolitics

  • “Every morning at six A.M., the first thing to hit my desk was this overnight report that included information on who got killed and what killed them,” O’Connell said. “I kept thinking, This is insane. Why are we in this place? It was not just Iraq but the whole picture. Why were we so invested in that part of the world?” The unsurprising answer that O’Connell came up with was oil. 2

Future

  • In Cowen’s future, huge gains in artificial intelligence will lead to the elimination of many of today’s high-employment lines of work. The people who thrive in this environment will be very bright and able to complement the machines and team effectively with them. As for the unemployed masses? Well, many of them will eventually find jobs going to work for the Haves, who will employ teams of nannies, housekeepers, and gardeners. If anything Musk is doing might alter the course of mankind toward a rosier future, Cowen can’t find it. Coming up with true breakthrough ideas is much harder today than in the past, according to Cowen, because we’ve already mined the bulk of the big discoveries. 2
  • If not, things will not improve by themselves. If you are kidding yourself, it is time to stop and plan for a jump. Barring any James Dean ending, your life is going to be LONG. Nine to five for your working lifetime of 40–50 years is a long-ass time if the rescue doesn’t come. About 500 months of solid work. How many do you have to go? It’s probably time to cut your losses. 8

Off the Cuff

  • (Musk nearly died on a trip to Africa, where he caught a type of severe malaria. It was discovered late on his return to the United States, and Musk spent 10 agonizing days in intensive care. It took him six months to recover and lost 20 kilos during the illness.) “I came very close to dying,” Musk said. “That’s my lesson for taking a vacation: vacations will kill you.” 2
  • “The guy comes in, and Elon asks him why they’re meeting,” Spikes said. “He said, ‘To develop a relationship.’ Elon replied, ‘Okay. Nice to meet you,’ which basically meant, ‘Get the fuck out of my office.’ 2



  1. Zero to One: Notes on Start Ups, or How to Build the Future (Masters, Blake;Thiel, Peter)↩

  2. Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future (Ashlee Vance)↩

  3. Steve Jobs (Isaacson, Walter)↩

  4. Methods of Persuasion: How to Use Psychology to Influence Human Behavior (Kolenda, Nick)↩

  5. Sapiens: A Brief History of Humankind (Yuval Noah Harari)↩

  6. Peter Pan (J. M. Barrie)↩

  7. 1984 (George Orwell)↩

  8. The 4-Hour Workweek , Escape the 9-5, Live Anywhere and Join the New Rich (Timothy Ferriss)↩

  9. Deep Work (Cal Newport)↩

  10. No More Mr. Nice Guy (Robert A. Glover)↩

  11. It Doesn’t Have to Be Crazy at Work (Jason Fried; David Heinemeier Hansson)↩

  12. Man’s Search for Meaning (Viktor E. Frankl)↩

  13. The Prophet (Kahlil Gibran)↩

  14. The Death of Ivan Ilych (Lev Tolstoy)↩

  15. The Brothers Karamazov (Fyodor Dostoyevsky)↩

  16. The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Change (Camille Fournier)↩

  17. The Five Dysfunctions of a Team, Enhanced Edition: A Leadership Fable (J-B Lencioni Series) (Patrick M. Lencioni)↩

The article’s thumbnail image background was created via Open AI’s DALL-E, with the prompt: “bright, lively and colorful epic light coming from piles of books, with pages flying, against white background, digital art”

]]> Learnings as a Software Engineer Techlead http://lopespm.github.io/2022/03/22/learnings-software-engineer-techlead.html http://lopespm.github.io/2022/03/22/learnings-software-engineer-techlead.html

YouTube video with a summary of this article

Life is growth. You grow or you die.

― Phil Knight, Shoe Dog

I’ve had the good fortune of having the opportunity to develop my software engineering work as a tech lead (which is not a fixed role), focusing more of my effort on solving sets of problems, as opposed to individual projects/tasks. While navigating this path, I’ve gathered a set of teachings from my mentors/peers/managers and other personal learnings and observations that I attempted to compile in a condensed list. Hopefully these can be useful in your own journey, just like other people’s learnings were to mine:

  • Planning:
    • Clear planning and prioritization of your projects is essential, especially in the early phases of an initiative. Well defined projects, with solid goaling metrics and well defined ownerships for each project, go a long way towards setting them up for success. These reduce the risk of not meeting goals, and allow for everyone involved to know which areas they need to tackle and own. Another interesting side effect is the value proposition to prospecting new-comers to support your projects, since there is a clear definition of what they can own and how valuable their contribution will be to the team/organization.
    • But don’t plan too much - healthy balance is the secret: if your organization is fast moving and dynamic, excessive planning can have the deleterious effect that little execution is actually done, since most of the time is spent planning.
  • Design Docs, Strategy and Vision:
    • Design Docs are a valuable vehicle to crystalize how a complex system or process can be implemented, they incentivize the owner to structure their thoughts, are made stronger by receiving feedback from peers, and serve as an exceptional piece of documentation that will not only explain how a given system/process works, but also serve a rationale on why they were built that way. Here is an example template from Google
    • Strategy and Vision - This reflection will be mostly based on this great StaffEng article, which I vividly recommend. If you realize that you’ve rehashed the same discussion three or four times, look into your organization’s objectives, find the commonalities from five of your organization’s design docs, combine all of these together, and you have material to write a strategy. When the future’s too hazy to identify investments worth making, grab five strategies, combine them with your organization’s objectives and write a vision for two to three years out.
  • Scaling oneself: The best way to increase impact is by scaling oneself. I’ve personally encountered my speed limit when trying to solve a critical-path project, while at the same attempting to set the direction for the team. As much as I worked, placed hours, or made the process effective, I was the bottleneck. I’ve hit a speed limit, and at the time it was made clear by my manager that in order to increase impact, unblock our team, achieve our objectives and have a balanced workload, I would need to scale myself.
    • Scaling through Delegation - the 75/25 rule:
      • If you are ~75% certain that someone will correctly execute a given project/task, then delegate. The other ~25% is the unknown space that the person needs in order to learn, explore, make errors and innovate. In other words, grow.
      • If you are 100% certain that the delegatee will successfully achieve a given project/task, they will not grow. On the other hand, if the probability of success is too low, then the project/task will be frustrating, probably will not be completed, and will jeopardize the team’s objectives.
    • By default, don’t micromanage:
      • When delegating a task/project, give enough space for the delegatee to own it. In general, the objective is to clearly define expectations, clearly define areas of ownership, and agree upon which details you should be kept up to date, which meetings you should attend, and in which situations you would expect to be pinged for escalations.
      • Micromanaging only makes sense in very specific situations, such as when there is a high possibility of a major failure and help is needed to firefight and course correct.
  • If you are not failing, you’re not pushing hard enough. And probably not growing: Conversely, the 75/25 rule can be applied to oneself. If you find you are easily executing your projects with a high level of predictability, this provides a hint that you should look for bigger challenges and responsibilities. In other words, aim for a growth mindset, rather than a fixed mindset.
    • Errors: Making errors is completely OK and expected. They are part of the ~25% above. Strive for them to happen only once, otherwise they will fail to serve their learning purpose.
  • Mentorship:
    • Give: one of the best ways to learn how to support projects is to mentor a new joiner. This will give you the opportunity to develop your interpersonal and project managing skills as you support/scope/prioritize projects and tasks for your mentee, and will help you consolidate your knowledge about the team and its position in the company as you transmit this knowledge forward. This relationship is especially rewarding in the long term as you work together on other projects down the line.
    • Receive: having good mentors is one of the best ways to quickly grow and learn. I have had the good fortune to learn from specially good mentors throughout my tenure, who apart from sharing their deep technical knowledge about our systems and processes, provided me solid foundations (either directly or by example) on how to navigate ambiguity and tackle challenging projects/situations. Every new mentor will have something novel to share and teach to you, it is an ongoing process. Good mentors provide you a golden path that you can follow to cut through the exhausting search for viable solutions to a given problem.
  • Learn from others:
    • Outside your organization: other people’s experiences are immensely helpful, specially the ones that have been time tested and are well structured. Books are a good source of this knowledge, and I would recommend The Manager’s Path, from which I extracted many of my base understandings of what a techlead should be, do and behave. The book goes straight to the point, gives actionable examples and was written by a very experienced manager.
    • Reach out to your peers: specially in the beginning of my role as techlead, I was struggling to find the right balance between personal impact from my own actions and (the less concrete) directional impact. To resolve my conundrum, I’ve reached out for support from my manager, my original mentor and a fellow experienced techlead. Their experiences and advices helped me to tackle this subject, and other challenges that came along the way.
  • Measure your success:
    • Goaling against quantifiable metrics = solid prioritization and justification: Make sure that all your projects are goaled against an objective, quantifiable metric. Otherwise it will be hard to prioritize between them and justify their value during planning and execution to your peers and external stakeholders.
    • Clear and quantifiable goals = increased motivation: Projects with goals based on solid metrics allow for anyone working on a project to know how valuable their contribution will be. I would claim that working on projects with an ambiguous value proposition is frustrating and will receive less attention and effort from everyone involved, since it will not be clear how impactful their actions will be.
    • Measure your personal progress by asking for candid feedback: if your manager and/peers don’t give you candid feedback on your progress, ask for it explicitly. It is one of the best ways to measure your progress on your role and surfacing any blindspots.
    • No measurement = flying blind
  • Communication - what you transmit, and how, matters:
    • Written and oral communication is essential when explaining the rationale behind an objective/project/task, and when coordinating with your peers and other stakeholders. This is a skill that can be learned, and one that I am still actively evolving. The best way to improve it, as I have found, is to write more, speak more, make more presentations, and study the communication methods from others.
    • As a rule of thumb, most communication styles benefit from a “news article” structure: first a TLDR (Too Long Didn’t Read), then an overall context of the problem in question and its impact, followed by details such as the delineation of a set of solutions, ending with a call to action or further questions.
  • Imposter syndrome: this might stem from a mismatch between your perceived performance and expectations for your role. Work with your manager to explicitly define expectations. Ask your manager and peers for explicit feedback, so you can better calibrate your perceptions. If you feel like an imposter, odds are, you aren’t.
  • Take care of your body and mind, they are your ultimate tools: Last but not least, exercise, eat healthy food, nourish your relationships and strive for quality sleep. These are the essential tenants for the infrastructure (you) that will sustain all of the above points.
]]> Hacker News Daily http://lopespm.github.io/2020/12/25/hackernews-daily.html http://lopespm.github.io/2020/12/25/hackernews-daily.html This article will dive into the system behind the generation of Hacker News Daily, an open source lightweight daily Hacker News best stories website, with screenshots and top comments.


Motivation

Hacker News is my go-to source for relevant, interesting and constructive discussions on a wide range of topics. I usually consume it via Daemonology’s Hacker News Daily, to catch up on the most active topics in the community.

Daemonology’s Hacker News Daily presents the title, story link, Hacker News discussion link, and is optimized for desktop. I usually consult it on mobile, and when I am several days behind, I sweep through the archives and open several story/discussion links in separate mobile tabs, in order to triage the stories with a quick glance at their web page and their discussion’s top comments.

This project aims to ease that process, by generating a set of web pages presenting the best daily Hacker News for mobile or desktop, with screenshots and top comments for each story, while aiming to have a low footprint to the end user.

Architecture

Information flow diagram between the different components of the system

The basilar idea is to recurrently get the best Hacker News stories via its API; take screenshots of the web pages that these stories link to; and have this workflow executed via GitHub Actions.

In specific, these are the steps taken to generate the final web pages (illustrated by the above diagram):

1. Every day, a new Github Workflow is spawned, kickstarting the entire process.

2. Update History

  • The first step is to get the past days best stories from the last successful run, and add the current best stories by reaching out to the Hacker News API to fetch the current list of best stories, removing the ones that were already featured, and adding the top ones to the current history
    • To persist the past best stories history, an days_history.dat artifact is created in every run, which is a simple Python Pickle containing the story ids from the past days. These days are stored in form of a deque, in order to pop the older days as the history grows larger
    • The final result of this step is a new days_history.dat, which will be used in the next step below, and will also serve as a base for the next workflow
    • If a viable days_history.dat artifact cannot not be found in the previous successful runs, it is rebuilt by parsing the Daemonology’s Hacker News Daily web page

3. Create Day/Story Models

  • From the story ids provided by the previously built days_history.dat, a list of hydrated days with their respective stories are built. These models will posteriorly provide all the information needed to create the final web page views
    • Each story is composed of an ID, title, story link, Hacker News discussion link, and its top comments. This information is obtained via the Hacker News API (example call)

4. Create the generated folder. This is where the generated web pages and screenshots will be placed, in order to be later deployed to GitHub pages

5. Gather Screenshots

  • Using pyppeteer, an headless browser is created to navigate through all the story links
    • A screenshot is taken for each of these pages (after attempting to dismiss a possible “Allow Cookies” prompt), and stored as PNGs in the generated folder
    • Each of these screenshots are re-encoded in WebP, which allows for a smaller image footprint in browser’s that support it
  • If there is an unexpected error while processing the screenshots, it will not halt the overall workflow, since the final web pages can still function without screenshot images

6. Generate Web Pages

  • Jinja, a templating engine for Python, is used to define and generate each of the final web pages. Four web pages are generated from a single template. These four variants are permutations of either presenting all past 10 days or not, and either showing images or not:
    • Latest day, with images
    • Latest day, without images
    • Past 10 days, with images
    • Past 10 days, without images
  • It was chosen to keep these in separate static web pages, in order to keep them simple, static, and without additional dynamic logic

7. The generated folder is deployed to the gh-pages branch, which is deployed as a GitHub page, making these generated contents publicly accessible

Source Code

The full source code is accessible at https://github.com/lopespm/hackernews_daily and the generated website at https://lopespm.github.io/hackernews_daily, feel free to improve it or to leave some feedback.

]]> Implementation: Autocomplete System Design for Large Scale http://lopespm.github.io/2020/08/03/implementation-autocomplete-system-design.html http://lopespm.github.io/2020/08/03/implementation-autocomplete-system-design.html This article will go through my implementation/design of a large scale autocomplete/typeahead suggestions system design, like the suggestions one gets when typing a Google search.

This design was implemented using Docker Compose1, and you can find the source code here: https://github.com/lopespm/autocomplete

YouTube video overview of this implementation

Requirements

The design has to accommodate a Google-like scale of about 5 billion daily searches, which translates to about 58 thousand queries per second. We can expect 20% of these searches to be unique, this is, 1 billion queries per day.

If we choose to index 1 billion queries, with 15 characters on average per query2 and 2 bytes per character (we will only support the english locale), then we will need about 30GB of storage to host these queries.

Functional Requirements

  • Get a list of top phrase suggestions based on the user input (a prefix)
  • Suggestions ordered by weighting the frequency and recency of a given phrase/query3

The main two APIs will be:

  • top-phrases(prefix): returns the list of top phrases for a given prefix
  • collect-phrase(phrase): submits the searched phrase to the system. This phrase will later be used by the assembler to build a data structure which maps a prefix to a list of top phrases

Non-Functional Requirements

  • Highly Available
  • Performant - response time for the top phrases should be quicker than the user’s type speed (< 200ms)
  • Scalable - the system should accommodate a large number of requests, while still maintaining its performance
  • Durable - previously searched phrases (for a given timespan) should be available, even if there is a hardware fault or crash

Design & Implementation

High-level Design

The two main sub-systems are:

  • the distributor, which handles the user’s requests for the top phrases of a given prefix
  • the assembler, which collects user searches and assembles them into a data structure that will later be used by the distributor

Detailed Design

This implementation uses off-the-shelf components like kafka (message broker), hadoop (map reduce and distributed file system), redis (distributed cache) and nginx (load balancing, gateway, reverse proxy), but also has custom services built in python, namely the trie distribution and building services. The trie data structure is custom made as well.

The backend services in this implementation are built to be self sustainable and don’t require much orchestration. For example, if an active backend host stops responding, it’s corresponding ephemeral znode registry eventually disappears, and another standby backend node takes its place by attempting to claim the position via an ephemeral sequential znode on zookeeper.

Trie: the bedrock data structure

The data structure used by, and provided to the distributor is a trie, with each of its prefix nodes having a list of top phrases. The top phrases are referenced using the flyweight pattern, meaning that the string literal of a phrase is stored only once. Each prefix node has a list of top phrases, which are a list of references to string literals.

As we’ve seen before, we will need about 30GB to index 1 billion queries, which is about the memory we would need for the above mentioned trie to store 1 billion queries. Since we want to keep the trie in memory to enable fast lookup times for a given query, we are going to partition the trie into multiple tries, each one on a different machine. This relieves the memory load on any given machine.

For increased availability, the services hosting these tries will also have multiple replicas. For increased durability, the serialized version of the tries will be available in a distributed file system (HDFS), and these can be rebuilt via map reduce tasks in a predictable, deterministic way.

Information Flow

Assembler: collect data and assemble the tries

  1. Client submits the searched phrase to the gateway via http://localhost:80/search?phrase="a user query"
  2. Since the search service is outside the scope of this implementation, the gateway directly sends the searched phrase to the collector’s load balancer via http://assembler.collector-load-balancer:6000/collect-phrase?phrase="a user query"
  3. The collector’s load balancer forwards the request to one of the collector backends via http://assembler.collector:7000/collect-phrase?phrase="a user query"
  4. The collector backend sends a message to the phrases topic to the message broker (kafka). The key and value are the phrase itself 4
  5. The Kafka Connect HDFS Connector assembler.kafka-connect dumps the messages from the phrases topic into the /phrases/1_sink/phrases/{30 minute window timestamp}5 folder 6
  6. Map Reduce jobs are triggered 7: they will reduce the searched phrases into a single HDFS file, by weighting the recency and frequency of each phrase 8
    1. A TARGET_ID is generated, according to the current time, for example TARGET_ID=20200807_1517
    2. The first map reduce job is executed for the K 9 most recent /phrases/1_sink/phrases/{30 minute window timestamp} folders, and attributes a base weight for each of these (the more recent, the higher the base weight). This job will also sum up the weights for the same phrase in a given folder. The resulting files will be stored in the /phrases/2_with_weight/2_with_weight/{TARGET_ID} HDFS folder
    3. The second map reduce job will sum up all the weights of a given phrase from /phrases/2_with_weight/2_with_weight/{TARGET_ID} into /phrases/3_with_weight_merged/{TARGET_ID}
    4. The third map reduce job will order the entries by descending weight, and pass them through a single reducer, in order to produce a single file. This file is placed in /phrases/4_with_weight_ordered/{TARGET_ID}
    5. The zookeper znode /phrases/assembler/last_built_target is set to the TARGET_ID
  7. The Trie Builder service, which was listening to changes in the /phrases/assembler/last_built_target znode, builds a trie for each partition10, based on the /phrases/4_with_weight_ordered/{TARGET_ID} file. For example, one trie may cover the prefixes until mod, another from mod to racke, and another from racke onwards.
    1. Each trie is serialized and written into the /phrases/5_tries/{TARGET_ID}/{PARTITION} HDFS file (e.g. /phrases/5_tries/20200807_1517/mod\|racke), and the zookeeper znode /phrases/distributor/{TARGET_ID}/partitions/{PARTITION}/trie_data_hdfs_path is set to the previously mentioned HDFS file path.
    2. The service sets the zookeper znode /phrases/distributor/next_target to the TARGET_ID

Transfering the tries to the Distributor sub-system

  1. The distributor backend nodes can either be in active mode (serving requests) or standby mode. The nodes in standby mode will fetch the most recent tries, load them into memory, and mark themselves as ready to take over the active position. In detail:
    1. The standby nodes, while listening to changes to the znode /phrases/distributor/next_target, detect its modification and create an ephemeral sequential znode, for each partition, one at a time, inside the /phrases/distributor/{TARGET_ID}/partitions/{PARTITION}/nodes/ znode. If the created znode is one of the first R znodes (R being the number of replica nodes per partition 11), proceed to the next step. Otherwise, remove the znode from this partition and try to join the next partition.
    2. The standby backend node fetches the serialized trie file from /phrases/5_tries/{TARGET_ID}/{PARTITION}, and starts loading the trie into memory.
    3. When the trie is loaded into memory, the standby backend node marks itself as ready by setting the /phrases/distributor/{TARGET_ID}/partitions/{PARTITION}/nodes/{CREATED_ZNODE} znode to the backend’s hostname.
  2. The Trie Backend Applier service polls the /phrases/distributor/{TARGET_ID}/ sub-znodes (the TARGET_ID is the one defined in /phrases/distributor/next_target), and checks if all the nodes in all partitions are marked as ready
    1. If all of them are ready for this next TARGET_ID, the service, in a single transaction, changes the value of the /phrases/distributor/next_target znode to empty, and sets the /phrases/distributor/current_target znode to the new TARGET_ID. With this single step, all of the standby backend nodes which were marked as ready will now be active, and will be used for the following Distributor requests.

Distributor: handling the requests for Top Phrases

With the distributor’s backend nodes active and loaded with their respective tries, we can start serving top phrases requests for a given prefix:

  1. Client requests the gateway for the top phrases for a given prefix via http://localhost:80/top-phrases?prefix="some prefix"
  2. The gateway sends this request to the distributor’s load balancer via http://distributor.load-balancer:5000/top-phrases?prefix="some prefix"
  3. The load balancer forwards the request to one of the frontends via http://distributor.frontend:8000/top-phrases?prefix="some prefix"
  4. The frontend service handles the request:
    1. The frontend service checks if the distributed cache (redis) has an entry for this prefix 12. If it does, return these cached top phrases. Otherwise, continue to next step
    2. The frontend service gets the partitions for the current TARGET_ID from zookeeper (/phrases/distributor/{TARGET_ID}/partitions/ znode), and picks the one that matches the provided prefix
    3. The frontend service chooses a random znode from the /phrases/distributor/{TARGET_ID}/partitions/{PARTITION}/nodes/ znode, and gets its hostname
    4. The frontend service requests the top phrases, for the given prefix, from the selected backend via http://{BACKEND_HOSTNAME}:8001/top-phrases="some prefix"
      1. The backend returns the list of top phrases for the given prefix, using its corresponding loaded trie
    5. The frontend service inserts the list of top phrases into the distributed cache (cache aside pattern), and returns the top phrases
  5. The top phrases response bubbles up to the client

Zookeper Znodes structure

Note: Execute the shell command docker exec -it zookeeper ./bin/zkCli.sh while the system is running to explore the current zookeeper’s znodes.

  • phrases
    • distributor
    • assembler
      • last_built_target - set to a TARGET_ID
    • distributor
      • current_target - set to a TARGET_ID
      • next_target - set to a TARGET_ID
      • {TARGET_ID} - e.g. 20200728_2241
        • partitions
          • |{partition 1 end}
            • trie_data_hdfs_path - HDFS path where the serialized trie is saved
            • nodes
              • 0000000000
              • 0000000001
              • 0000000002
              • …
          • {partition 2 start}|{partition 2 end}
            • …
          • {partition 3 start}|
            • …

HDFS folder structure

Note: Access http://localhost:9870/explorer.html in your browse while the system is running to browse the current HDFS files and folders.

  • phrases
    • 1_sink *- the searched phrases are dumped here, partitioned into 30min time blocks, *
      • {e.g 20200728_2230}
      • {e.g 20200728_2300}
    • 2_with_weight - phrases with their initial weight applied, divided by time block
      • {TARGET_ID}
    • 3_with_weight_merged - consolidation of all the time blocks: phrases with their final weight
      • {TARGET_ID}
    • 4_with_weight_ordered - single file of phrases ordered by descending weight
      • {TARGET_ID}
    • 5_tries - storage of serialized tries
      • {TARGET_ID}
        • |{partition 1 end}
        • {partition 2 start}|{partition 2 end}
        • {partition 3 start}|

Client interaction

You can interact with the system by accessing http://localhost in your browser. The search suggestions will be provided by the system as you write a query, and you can feed more queries/phrases into the system by submitting more searches.

Source Code

You can get the full source code at https://github.com/lopespm/autocomplete. I would be happy to know your thoughts about this implementation and design.



  1. Docker compose was used instead of a container orchestrator tool like Kubernetes or Docker Swarm, since the main objective of this implementation was to build and share a system in simple manner.↩

  2. The average length of a search query was 2.4 terms, and the average word length in English language is 4.7 characters↩

  3. Phrase and Query are used interchangeably in this article. Inside the system though, only the term Phrase is used.↩

  4. In this implementation, only one instance of the broker is used, for clarity. However, for a large number of incoming requests it would be best to partition this topic along multiple instances (the messages would be partitioned according to the phrase key), in order to distribute the load.↩

  5. /phrases/1_sink/phrases/{30 minute window timestamp} folder: For example, provided the messages A[time: 19h02m], B[time: 19h25m], C[time: 19h40m], the messages A and B would be placed into folder /phrases/1_sink/phrases/20200807_1900, and message C into folder /phrases/1_sink/phrases/20200807_1930↩

  6. We could additionally pre-aggregate these messages into another topic (using Kafka Streams), before handing them to Hadoop↩

  7. For clarity, the map reduce tasks are triggered manually in this implementation via make do_mapreduce_tasks, but in a production setting they could be triggered via cron job every 30 minutes for example.↩

  8. An additional map reduce could be added to aggregate the /phrases/1_sink/phrases/ folders into larger timespan aggregations (e.g. 1-day, 5-week, 10-day, etc)↩

  9. Configurable in assembler/hadoop/mapreduce-tasks/do_tasks.sh, by the variable MAX_NUMBER_OF_INPUT_FOLDERS↩

  10. Partitions are defined in assembler/trie-builder/triebuilder.py↩

  11. The number of replica nodes per partition is configured via the environment variable NUMBER_NODES_PER_PARTITION in docker-compose.yml↩

  12. The distributed cache is disabled by default in this implementation, so that it is clearer for someone using this codebase for the first time to understand what is happening on each update/step. The distributed cache can be enabled via the environment variable DISTRIBUTED_CACHE_ENABLED in docker-compose.yml↩

]]> Survival Ball: Making the Game http://lopespm.github.io/2019/02/06/survival-ball-making-the-game.html http://lopespm.github.io/2019/02/06/survival-ball-making-the-game.html

YouTube video version of this article

Before: Initial prototype, built 6 years ago

After: Current version, released last November

This article/postmortem provides an in-depth look into the process of building Survival Ball, a Single / Local Co-Op physics-based game available on Steam for Windows and macOS. From prototype until showcase at Lisboa Games Week, passing by the related principles, design decisions, level creation process, tools and technical details.



Prototype

About six years ago I started learning Unity in my spare time, going through several official and unofficial tutorials. One was a simple tutorial on how to create a wire chain through physics. After its completion, I added a simple platform and a sphere. When arrow keys were pressed the sphere was torqued, and on spacebar press a vertical force was added to sphere, which caused it to jump. I was at awe. Unity’s physics were accessible and felt natural out of the box. Not only that, but a rough working scene could be quickly setup by gluing some basic concepts.

Wire chain and sphere physics

An idea started to bubble up about around this simple project: staying as long as possible on top of a platform. Antagonists, materialized in the form of various hazards, were added in order to coerce (directly or indirectly) the player off the platform. Frequency and power of these hazards increased over time, raising the probability of reaching the level’s finishing state. From this starting point, more elements were added, such as interactable platform edge blocks, destroyable floor tiles, backdrops, user interface and basic sound effects.

Two versions resulted from this. The first ad supported version was available for Android mobile devices only, having one playable level in single-player mode. Afterwards, a new adless version was released for OUYA, Android TV and Kongregate, featuring a new versus mode.

Game prototype version for OUYA, Android TV and Kongregate

Reboot

I went to work on other projects after the prototype’s launch, but the feedback provided from gamers was kept in the back of my mind in the meanwhile. Several ideas eventually piled up for a new, improved version of the game, and about two years ago they were first brought to fruition. First at night after work, then full-time until completion.

Laying out a solid base was was essential in the beginning of the project, specially concept wise. Pre-production, if you will. Surviving on top of a platform was the root idea, but was there latitude for perpendicular ideas? Multiplayer mode? Versus or Cooperative? Online or local only? Powerups? Which player movements would be available, only jump? Progressive difficulty (the player does not choose the difficulty level) or bucket based difficulty (e.g. easy, normal, hard)? What will be the direction of visual style, audio, and code architecture? How many levels? What is the player’s motivation play these levels?

Root ideas and game modes

Less is more when it comes to the game’s root ideas, and surviving on top of a platform seemed like a good premise. This idea was dissected and questioned thoroughly during the initial phase of development, in order to make sure it was well sustained. Other ideas such as going as high as possible or objective based levels were considered, but set aside in the initial stage of the project. They were not completely scrapped, such that both survival/time based and objective based levels would later cohabitate throughout the game’s development.

Assuming survival as the most important root idea, a question came up: survive what? It could be either you against the world, or you against other player(s), or a combination of both. The latter two options would prompt the need of a multiplayer versus mode and/or player AI, and the former one would naturally accommodate a single player and/or cooperation mode.

Allowing for a local multiplayer mode would be something desirable, since I had a personal bias towards games that physically join a group of friends in the same place. Also, online multiplayer was set aside due to the added entropy and complexity associated with it.

As for cooperative (co-op) games, there is usually an increased inclusion factor due to the wealth of opportunities to strike a balance between experienced and less experienced players, translating to less chances of frustration caused by huge proficiency gaps.

Given the above points, solo and local co-op modes were chosen to be included in the game.

Difficulty

Again, having in mind that less is more, a progressive difficulty mode (where the player does not choose the difficulty level) was strived for, instead of a bucket based approach (e.g. easy, normal, hard). Choosing a difficulty level when starting a simple game like this one, forces the player to commit to a given difficulty level at start, and accepting that decision. The responsibility is on the player. Often happens to me that when I choose a difficulty level like “easy” or “normal”, I keep wondering throughout the game if I am really good at that game, or am I just being thrown softballs. Progressive difficulty resolves that problem immediately, but it falls on the developer shoulders to craft the game in such a way that all difficulty ranges are covered just right.

Progression would be 2-fold: each level would start in an accessible way, but would gradually get more difficult as time passes. On the other hand, each level would be more challenging than the last one during the campaign.

Control and movement

There was the need to have offensive manuevers against hazards, and to provide mid-air control to the player. Two new movements were added to the already existing jump and double jump movements: dash and stomp. Dash provided mid-air control and allowed the player to interact with the world’s objects and attack hazards. Stomp, inspired by Super Mario 64, allowed for button interaction, and mid-air stop and plunge for precise vertical landings.

With this set of movements, the player was able to gain full control over the ball. Easy to learn, hard to master. Additionally, it suppressed the need for aerial control via directional controls, which would be less physically coherent and could quench some of the fun of gaining control proficiency.

Powerups

An assumption was made that powerups would be distributed randomly, either by items scattered throughout the level or by other means. Powerups were not used because of these four reasons:

  • Added game design complexity - every new powerup would increase the realm of possible scenarios during a game session, which would increase the difficulty of striking good game balance.
  • Simplicity - simplicity was held at high value during development, and powerups would probably require the usage of a new button to trigger the power, more UI elements and new concepts for the player to grasp.
  • Skill based gameplay - the game aimed to be purely skill based. Since the randomized nature of hazards already abducted some of this intent, having powerups attributed by chance would further decrease the player’s perception that the level was completed by his/her own merit.
  • They could be used as crutches - powerups are outstanding when used properly, but can also be abused to cover up design flaws. Having a no-powerup limitation forced me think in alternative ways to make the game more interesting, a challenge I was drawn to.

Visual Aesthetics

Defining the game’s visual appearance early on was quite important, since it would influence the reasoning behind 3D models, textures, UI, colors, mood and even audio. The two most influential references were Material Design (first version) and Lara Croft GO.

Coming from an Android development background, I was exposed to Material Design, which is a visual language that synthesizes the classic principles of good design. Its principles go beyond mobile UI development, and span to a swath of different form factors and use cases. I personally found it beautiful and applyed its principles throughout the game’s aesthetics, both in UI, but also in the overall style of the game. This influenced the game to have a simple, clean and coherent style.

Lara Croft GO is a beautiful game. The charming colors and elegant (yet simple) style made an impression on me. While building up the game’s aesthetics, I consulted several screenshots of Lara Croft GO and attempted to transpose its most pleasant design elements into Survival Ball.

Codebase

The prototype was a literal extension of the initial wire chain and ball physics experiment, and features were added at the same time I learned Unity’s ropes. This, added to rapid prototyping and little to no code architecture planning, resulted in a codebase that was entangled, difficult to follow and hard to extend.

The new version was built from scratch and several arquitectures were explored in the beginning of the project, until one was found to fit the game best. The chosen approach followed these principles:

  • Each level had a single entry point, the game controller, which orquestrated the different actors.
  • The game controller contained all top level parameters. Sub-parametrization was facilitated via Scriptable Objects. For example, the PlayerController had PlayerParameters (which was a scriptable object) exposed on the game controller. The PlayerParameters could in turn have sub-parameters, if needed.
  • Dynamic objects, such as the player, hazards and interactable level parts should be created programmatically whenever possible.
  • The initial scene stage should only have static or decorative objects.
  • Object behavior should be managed by its specific controller. Game object components should be presentational and have as little behaviour as possible. For example, a rocket thruster could have a component which changed the color and size of smoke and fire, but it would be the thruster’s controller responsibility to drive the input value for this component.
  • Orquestration of events (from dynamic objects, user interaction and other data streams) was handled using a reactive approach, facilitated by UniRx

Levels

The level creation process went somewhere along these lines: I would go for a long walk without any audiobooks or entertainment, and allow my mind to wander around. If an idea came up, I would write it down in a Google Keep note. At a later time, in a quiet place, I would dissect these ideas on paper to materialize its pros and cons, while further developing the ideas that seemed most promising.

First Sandbox

Before building any of the levels, a scrappy sandbox level was built to work out the broader strokes of the player’s movement. In this sandbox, movement, jump, stomp and dash (immediate single charge at the time) were first developed and loosely tweaked.

RocketX

SpaceX was landing their first rockets on drone ships when the first idea of the bunch was developed. It revolved around the concept of a four thruster rocket platform. Each of these thrusters could be activated or deactivated by their respective button. An additional center button turned all the thrusters on or off. The platform had a finite amount of fuel that could be replineshed by shoving fuel crates, hazards or players in the center fuel intake.

This level would set the tone for the upcoming ones, so it was important to pin down the core gameplay, level structure and hazard dynamics before moving on to the next level. Just like changing a cosmological parameter would drastically change everything in the universe (like the possiblity of life), the game’s basilar parameters were set as tightly as possibile early on, because even a small change could break other already built levels. For example, a small change in how fast the player is, how high can it jump or how gravity affects it, could translate into several hours of extra work to re-adjust every level accordingly, or even render them obsolete.

For these reasons, it took around 354 hours to develop the bulk of RocketX, about one fifth of the project’s timespan.

Initial RocketX prototype, to access the
overall feel of the level

Textures, props, backdrop and hazards added,
along with several adjustments to player gameplay

Four players playing the final version

Garfunkel

Original concept for the second level was a variation of the Simon game, hence Garfunkel. A sequence of lights would be played out first, and the player(s) would then have to mimic the same sequence afterwards by orderly hitting the respective target pieces, at the cost of the piece falling off if it were to be hit outside the sequence. There would exist as many platforms as the number of players.

A quick level layout was built. Soon after the first plays and it became apparent that the concept was flawed. Mimicking a sequence would mean that during the first phase, while the model sequence is played out, the player would be left to do nothing movementwise. Moreover, after the second or third prompt of the sequence, the player would most likely forget which came next, because his/her main focus would be getting the ball movement right and not hit any of the non sequenced tiles. Simon is about memory, Survival Ball is about motor skill.

Initial Garfunkel prototype, exploring a variation of the Simon electronic game

Pivoting from this, came the idea of collapsing the two phases into a single phase, no memorization needed. The platform piece prompts would be immediate and coupled with the background music rhythm, which would progressivly get faster as time passed. The player would have limited time to hit these prompts, as the unhit prompted pieces would fall after each music measure.

There was only one platform, since having multiple platforms would elicit a natural exploit. The player could quickly notice that the best strategy would be to focus all energy in a single platform. There was no real gain to be had on saving all platforms. One would suffice, and would be easier for the player to manage.

The background music was programmatically played. The easiest approach was to individually play each audio sample, meaning that the rhythm was dictated by the frequency to which these were played. Long and complex audio samples were not fit for this purpose, since they had their own tempo, and only transforming these samples would correctly match them with the overall music tempo. Simple and short audio samples were used, specially drum samples, which worked great for this use case.

1. First tribal backdrop experiment

2. Waterfall backdrop, later re-used in Unfair Fair

3. Level exploit,
bypassing piece decay, platform rotations and prompts

4. Simple particle system backdrop,
synchronized with the music

The music engine and platform movement were first tackled, and took a considerable amount of time to build. After a few playthroughs with these basic systems in place, something did not feel quite right. At the time, it seemed related with how the background looked like. Predominantly using drum samples, the overall music style suggested a tribal feel, and seemed important at the time to have a good match between the music and the level’s visual appearance, hence the different backdrop experiments.

After these four different backdrops, something was still off. Progressively I realized that the real problem was related with the platform’s shape. In a circular platform, the best strategy is to safeguard the inner pieces, since they are easily reachable and have an inferior angular speed compared to the outer ones. Falling prey to sunk cost fallacy, some attempts were made to save this concept: rotate the inner pieces at a higher speed or to rotate them in different axes; to implement piece decay, which released the piece if the player stayed on top of it for too long. None of them worked, since these obstacles could be easily bypassed, as shown by the above video number 3.

Final Garfunkel state. The level was scrapped but most of its elements were re-used in Beatmill and Big Giant Head

The last backdrop experiment was a simple water floor, which focused the player’s attention in the platform and had the additional benefit of having a near plane to which the player’s shadow could be casted. The vertical player shadow was a very important reference for the player’s position.

The level would need to be completely redesigned or scrapped altogether, since it was not acceptable in its current form. It ended up being scrapped, but the waterfall backdrop (video 2. above) was later reused by Unfair Fair, and two new levels were forked from Garfunkel’s base elements: Beatmill and Big Giant Head.

Beatmill

Beatmill addressed one of the major issues in Garfunkel, the easy exploit of the circular platform shape. Solution was to have several loose square pieces which were part of a cross or lengthwise treadmill. Randomly moving each piece along these two directions meant that no piece was permanently in the same area (like the center area) of the platform. Every single piece was important to treasure and save.

The treadmill system was developed from scratch, and Garfunkel’s music engine and water floor backdrop were reused.

Single piece treadmill logic,
which would generalize in to a treadmill system

Treadmill system

After developing the level’s basic game elements and validating their gameplay via play testing, the decay concept was recycled from Garfunkel, with slight modifications. A special black button was placed as one of the platform’s pieces. If pressed, it decayed the adjacent pieces. If a decayed piece was touched, it would fall.

This concept apparently worked well, but when play tested with a group of friends, it was pointed out as being too punishing. When the special black button was pressed, the level’s difficulty skyrocketed, making the already difficult task of reaching prompted pieces even harder. As a result, pressing the black button was faced as a death kneel, urging players to restart the level right after the black button was pressed. After a few rounds of these, collective despair would soon grow due to the overwhelming difficulty.

Solution was to invert the black button’s function. Instead of decaying the adjacent pieces, it would revive them. This time around, the special black button was not avoided in fear, but in greed. It was a precious resource, to be used as a late as possible.

Four players playing an early version,
with the special black button decaying the adjacent pieces

Four players playing the final version,
with the special black button reviving the adjacent pieces

Big Giant Head

Garfunkel’s platform did not work in a survival setting, but could it work in a different setting? In order for it to work, the level’s progress could only advance when the player touched his/her respective prompt. An objective sequence. The level also needed a long term objective for these sequences to link up. Turns out the game campaign was lacking a final boss, and this could be a good opportunity to salvage Garfunkel’s platform (and its respective movement and prompt system) to a new level, the final one. The final boss.

The defined long term objective was to deplete the boss’s life, but the players needed a way to interact with it. The first idea was to drain the boss’s life every time a prompted platform piece was hit, but that bore two problems: interaction was too indirect and there was no opposite force that gave the boss a chance to defend itself. Hazards could be used to fulfil this purpose, but that dynamic was too easy and insipid. The final boss was expected to be more challenging than all its previous levels, and was expected to require the player to use most, if not all, of the previously aquired learnings.

Inspired by classic game bosses, which stack different boss stages, solution was to materialize the boss into an anthropomorphic head, and have the head appear in the center, equipped with a turret, when it was sufficiently small to fit platform’s center gap. The head shrank every time a sequence of prompted platform pieces were hit. The players could directly interact with the center head by hitting it, draining the boss’s life. Difficulty progression was accomplished by decaying a set of platform pieces every time the small center head was defeated.

Interactable boss turret prototype
added to the Garfunkel platform

Three players playing the final version

Unfair Fair

“Giant washing machine drum” was the initial concept behind this level. Challenge was to transform this germinal idea into a fun level, coherent with the game’s survival motto. Some scattered ideas were drafted on paper, such as having interactable controls to make the drum rotate in a given direction or to stop the rotation entirelly, but none of them were successfuly transformed in to a viable challenge or part of a larger challenge.

Hardcoded drum to validate to overall feel of the level

The most interesting avenue was to reuse the piece decay concept from Beatmill. In Unfair Fair, the decay was caused by a specific hazard, which if not hit by its respective player, exploded and decayed the nearby platform pieces. Once these pieces were touched, they fell. The pieces near the platform spokes were decayed at start, to increase the chances of partial or complete piece group detachment.

The first iteration of this hazard exploded after bouncing a given amount of times, which proved to be unfair for when the hazard bounces close to the ground, giving little opportunity for the player(s) to react.

The next and final iteration had the hazard explode a given time amount after its first bounce, which gave a fair chance for its dismantling. Visual presentation was reviewed to better present how close the hazard was to explode. The time taken to explode would decrease as the level progressed.

Notice the waterfall backdrop below, repurposed from one of Garfunkel’s experiments.

First version
Decay hazards exploding after a given amount of bounces

Four players playing the final version
Decay hazards explode after a given amount of time

Venom Rig

Upon Garfunkel’s closure, began the development of a concept revolving around an oil rig shaped platform. The platform had four pillars, each destroyed when their respective button was pressed. The player had no motivation to press them, but these could be pressed by stomper hazard developed previously for RocketX. Their only task was to press these buttons.

To add another challenge dimension, a concept from the game’s prototype was added: decay rockets (in the prototype they were represented as rectangular parallelepipeds). Upon landing, these hazards decayed the platform piece below until the piece was completely detached from the platform. Other hazards would be spawned throughout the lifecyle of the level for increased diversity.

First iteration of the level
with a dynamically generated platform

Added decay rockets and other hazards,
along with model, backdrop and texture work

Four players playing the final version

Hex Elevator

The initial concept on paper was a 2.5D level in which a fluid (water/lava) would progressively rise as the player(s) made their way up a series of platforms and objects. The rising fluid concept seemed promising, but having a series of fixed 2.5D platforms seemed somewhat bland and left little space for cooperation dynamics.

Solution was to have an infinite amount of procedurally generated platforms, each higher than the previous one. The only way to reach the next platform was via an elevator assigned to a specific player, which first had to be activated by touching all of its respective prompts. The elevators would only rise when all players occupied their respective elevator.

Reb Blob Games’ guides on hex grids were immensly useful when building the procedurally generated honeycomb platforms. These guides couple well explained theory with concise practical implementations, and I highly recommend them to anyone doing hex grid work.

First iteration of the level with procedurally generated
platforms and their respective prompts and elevators

Added rising water which was
replaced by lava in the upcoming iteration

Upon the first play test session with friends, it became apparent that small tweaks were required to the behaviour of elevators. Because elevators would only rise when all players were sitting on top of their respective elevator, a problem arised if, near the upper platform, someone abandoned the elevator earlier than others. At that point, all elevators would drop, taking most of the players with them.

Solution was to make each elevator independent when they were close the upper platform, allowing for a smoother team transition between platforms.

Four players playing the early version without
elevator independence upon reaching the upper platform

Four players playing the final version with
elevator independence upon reaching the upper platform

Tutorial

Is is generally a good idea to write the introduction of a book, essay or paper last, since you need to have a very solid idea of the shape of the finished product, and exactly what you need to mention up front for everything to hold together. The tutorial level was left for last for the same reasons.

It was the game’s first level, and had the responsability to introduce the game and provide the required tools and knowledge to play it. Specifically, how to jump, double jump, dash and stomp. The overall concept of the level was inspired by an early version of Gang Beasts’ tutorial, which was pedagogic andp fun. Having the controls layed out as part of the scenario, instead of using an UI overlay fixture, was something quite interesting as well.

First iteration. Functional, yet bland

The first iteration had some of the required pedagogic elements, but was static and insipid when played, even though it had some interactive elements.

The following idea was to have a series of small platforms sitting by a pond. The instability of these floating platforms imprinted a livelier dynamic and were more physically sound than the initial tutorial iteration.

Second iteration, featuring
several floating platforms inside a pond

Refinements to the second iteration

After a play test with friends, it was noticeable that very few people understood how to dash, or how important the combination of jump, dash and stomps were to gain air control.

Solution was to add an animated billboard ilustrating how the dash charge worked, and add two additional sections focused on air control. The following play tests showed that these changes made a significant difference on a player’s comprehension of the game’s basilar movements and techniques.

Final iteration with an animated dash charge billboard and new sections focused on air control

Extra: Block Towers

This level was not used, since no plausible game dynamics were found to fit the overall concept of the game, but the sheer fun and simplicity it provided while moving around high stacks of blocks were worth the special mention.

Most of these interfaces were built in the late stages of development. Following the Visual Aesthetics layed out earlier, the first rough screens were sketched out, starting by the home screen. Some experiments were made with the game’s logo and UI elements. UI buttons design for example, referred their colors and shape from a core element of the game, the hexagonal wave counter.

First iteration of the home screen

At about the same time, the title screen was built. Presented when opening the game and right before the home screen, it was crafted to create an impactful first impression and to be later used for key art, showcasing the most important elements of the game: logo, players, enemies and some levels. Another important aspect was to illustrate that this was a co-op game, so the 4 players were placed in the forefront, aligned in such a way that no one was widely in front of anyone, with about equal highlight to all of them. The screen drew inspiration from other games’ key art, such as the one from Super Mario Land 2, which presented all game’s key elements in one powerful, stylized image.

Title screen

The home screen was later refined and the remaining screens were progressively built and polished. These were the options (video, audio and controls), player selection, pause, end game, game selection and stats screens.

Tour of all menus and graphical interfaces

Audio

Music and sound effects were built using GarageBand. Many freely available sound libraries such as the Sonniss GameAudioGDC Bundles were used to mash up different samples into new sound effects.

GarageBand project for the main theme

Audio aimed to be clean and coherent, thus a small subset of samples were used from the immense libraries available and reused whenever possible, trying to settle with a mostly electronic style for music and to use instruments or natural audio effects for SFX.

Keys

I believe coherency is gained when using a small set of musical keys throughout the game. Survival Ball only used the C major key and its relative minor, A minor. Major keys are genererally recognized as happier and joyful and minor keys as darker and heavier. In the game, each level or screen used one of these keys, according to the desired environment and feel.

One interesting nugget of knowledge I came to learn while developing the audio, was that sound effects also fit within a certain key. The overall audio will sound much better when sound effects and music are built in key accordance.

EQ

Another interesting tidbit was to remove the extremely low and high frequencies in the final mix. The effect is threefold: audio fatigue is reduced when editing; makes the overall composition seem cleaner to the listener; leaves more headroom in the audible frequency range, improving the composition’s focus and clarity.

Extremely low and high frequencies cutoff in the master bus’ EQ

Organization

The game ended up having more than 60 different sound effects and around 20 musical segments. A considerable amount of time was wasted checking on a given audio segment status, so a simple spreadsheet was used to track the name, category and its state (done, needs revision, missing). This simple sheet improved the audio creation process considerably.

Play Tests

AI agents

Having the need to perform many iterations and tweaks without constant access to other testers, a simple AI was built to simulate multiplayer dynamics and to have a better grasp of how the level handles these dynamics.

AI agents were specific to each level, and driven by a series of objectives. For example, in Beatmill, the agent’s main objective was to reach its respective colored prompt. If no prompts existed, it would move towards the platform’s center, the optimal place to wait for a prompt. It also had passive behaviours such as avoidance of gaps or special black buttons via jumps. As seen in the video below, the agents are not perfect, but set a good work baseline.

Beatmill (non published early version): the blue, green and yellow players are controlled by AI

User Tests

Once roughly tweaked, the game was presented (preferably for the first time) to a group of friends during a play test session. These sessions were of uttermost value, since they provided valuable feedback, critics and/or validation to the various game elements.

There were 6 play test sessions in total. Apart from how valuable these were to improve the game, they were also a great opportunity to gather a group of friends and have a good time. The game was modified and tweaked between each session, after digesting the impressions and feedback of the previous one.

The game had some backdoor hooks which allowed for quick macro difficulty tweaks, e.g. how many waves were necessary to unlock a level. These were specially useful in the first play session, since the game was extremelly punishing in its early incarnations.

Some guidelines I tried to follow:

  • Refrained from playing. I only joined if the backdoor hooks were not enough to lower the difficulty to a point that was enjoyable by the players. I could infrequently join near the session’s end and most of the feedback had been provided.
  • I sat behind the players, writing impressions and feedback. Sporadically I asked for more details on a comment or behaviour that I observed. Being an observer, it is also easier to probe for mood fluctuations in the room. For example, when frustration starts to build up due to the extreme difficulty of a level, it is easily noticeable when observing from outside. The same applies for bursts of joy a given level or dynamic might cause.
  • Gut reactions were very precious, so I would probe for more details on these reactions and encourage honest reactions and feedback.
  • After the game was finished, I asked the players about their impressions, what they liked, they did not like, the level they enjoyed the most and the level they enjoyed the least. On that note, RocketX, Hex Elevator and Big Giant Head were top favorites. Venom Rig, Unfair Fair and Beatmill were most often referred as the less enjoyable of the bunch.
  • Only explained the reasons behind a given design decision after the game was finished and most of the feedback had been given, to diminish the influence of my opinion on the players feedback.

Launch

The game was launched on November 8th, 2018. One month earlier, the Steam store page, official website and twiter account were brought online. During that month, a closed beta ran on Steam, which served mostly for last mile validations, since the bulk of the iterations were made during the in-person play testing phase. At the same time, several dozens of keys were sent to youtubers, with notable regard to the ones specialized in couch co-op games. To the ones contacted by email, I took into attention this video by Stephen, where he describes what kind of emails he expects to receive from developers.

After launch, I was advised by fellow developers that the above strategy was not optimal, because it is usually a good idea to build your following by sharing your progress and interacting with the community during development (via social networks or dev blogs for example), potentially increasing the game’s exposure and sales. In a possible new game, it would be interesting to adopt this strategy and take note of how workflow and game design would be influenced.

Indie X

Shortly after completing the first beta build of the game, I submitted my application to Indie X, the biggest indie game showcase/contest in Portugal. Fortunately, Survival Ball was accepted as one of the 55 finalists, meaning that it would be showcased at Lisboa Games Week 2018 a week after the game’s launch!

A custom build was specially crafted for the event. To better fit the event’s environment, the build offered an arcade experience through local leaderbords for the players with the highest number of completed waves, and leaderboards for the fastest players to finish Big Giant Head or the Tutorial. A simple controls cheat sheet was added to the pause menu, and the end game screens were changed to allow the players to enter their (group) name to the leaderboard. All levels were unlocked in this build, avoiding the need to finish the campaign to access a specific level.

Highlights of the custom build for Lisboa Games Week 2018 showcase

The event surpassed all my expectations. It was the first time I saw, in person, swathes of anonymous people playing Survival Ball. It was pleasantly surprising to recurrently see many groups of gamers trying out the game, playing for hours, and even returning back to the booth at a later time. Not much sleep was had during those four days, but it was rewarding to witness such moments.

Some of the players at Lisboa Games Week. More photos and videos here

The showcase was also a great opportunity to connect with other developers and get to know their stories and games. The overall experience was incredible and I am immensly thankful to the organizers for putting it all together 🙏

Additional Notes / Appendix

Tools

  • Unity - game engine in which the game was built
  • Rider - .NET IDE used to develop and manage the game’s code
  • Blender - used to build all the 3D models and to map their respective textures. Blender is one of the most powerful FOSS tools I have ever used, and puts up a good fight against proprietary packages such as 3ds Max
  • Inkscape - all textures were crafted in Inkscape. Almost all textures were vector based, and later rasterized for Unity in PNG format
  • GarageBand - music and sound effects composition
  • Tyme - project time tracking
  • Git - source control. GitLab private repositories were used to host the game’s project and website

Time log

Time was tracked during the entire span of the project, in various categories.

Time for all categories, per milestone:

  • Until first complete alpha build - 1334 hours
  • Until Steam release - 1642 hours
  • Total, until now - 1716 hours

Time until now, per category

  • Coding, visual work, tooling, research/study - 1417 hours
  • Audio (music, sfx) - 92 hours
  • Final builds play tests (does not include play tests with other people or tests during development) - 32 hours
  • Website - 22h
  • Marketing, screenshots, videos, social media - 123 hours
  • User support - 8 hours
  • Administrative, burocracies - 22h

Final Notes

  • The game was developed in macOS. Windows was mostly used to work out the particularities of its respective build, such as build post-processing work.
  • In case you need one more reason on why to use version control for your game project: time travel. The game was under version control since day one, and its git repository has around 1.3K commits at the time of this writing. Going back in time was as simple as checking out an older commit. Almost all videos presented throughout this article were made possible because of this. Checkout, open project, record, repeat. Not only that, but it was fairly easy to see how work evolved over time, which helped immensely when writing this article.
  • Most challenging aspect was to make sure everything was coherent. Once root concepts, design and objectives were aligned and the early level prototypes felt good when played, then filling out the gaps (refining gameplay, models, textures, audio, UI) was somewhat straightforward.
  • The game’s development could be divided in three phases:
    • First phase (until first alpha build) - bulk of the development work was done during this phase. Very intense in terms of making sure everything was coherent, sane and enjoyable to play. There were large temporally contiguous blocks of work on each level.
    • Second phase (first alpha to public launch) - development was scattered, moving from place to place, iterating where needed and finishing up peripheral features such as Steam integration.
    • Third phase (from launch onwards) - maintenance work, user support, optimizations and bug fixes.
  • I had an amazing experience building this game and am very happy with the final result. Not only did it materialize one of my dreams, but gave me the opportunity to explore and deepen areas outside my professional expertise, such as 3D modeling/texturing, audio production and game design. If you are a game developer or are thinking on becoming one, I hope this testimony was useful for your journey and hopefully brought more clarity into the process 🙌
]]> Unity: Dynamic Multi Target Camera http://lopespm.github.io/libraries/games/2018/12/27/camera-multi-target.html http://lopespm.github.io/libraries/games/2018/12/27/camera-multi-target.html

Mostly invisible, yet essential, camera work is key to any game with dynamic cameras. This article dissects a concise Unity open source library which dynamically keeps a set of objects (e.g. players and important objects) in view, a common problem for a wide range of games.

The library was developed for, and used by my first Steam game, Survival Ball. The game has an heavy shared screen local co-op component, which requires the camera to dynamically keep many key elements in view.

There are good camera resources for Unity, but I found them to either do too much or too little for this specific problem, so I thought this could be a good opportunity to learn a bit more about dynamic camera movement and to share that knowledge and code with the community.

Overview

The library is fed with the desired camera rotation (pitch, yaw and roll), the target objects that will be tracked and the camera that will be transformed.

The library’s sole responsibility is to calculate a camera position in which all targets lie inside its view. To achieve this, all target objects are projected onto a slice (plane) of the camera’s view frustrum. The projections located inside the view frustrum will be visible and the others will not. The idea is to trace back a new camera position from the outermost target projections, since this way we are guaranteed to include all projections inside the view.

In order to make the bulk of the operations easier to compute, the process starts by multiplying the camera’s inverse rotation with each of the targets positions, which will place them as they would if the camera’s axis would be aligned with the world’s axis (identity rotation). Once the camera position is calculated in this transformed space, the camera rotation is multiplied by this position, resulting in the final desired camera position. The actual camera position is then progressively interpolated towards this desired position, to smooth out the camera movement.

Original targets positions
Multiply the camera’s inverse rotation with each of the targets positions
Calculate the camera position in this transformed space
Multiply the camera’s rotation with the calculated position in the previous transformed space, which will reveal the final camera position in the original space

Implementation

Most of the operations are performed in the transformed space where the camera’s axis would be aligned with the world’s axis (identity rotation). After the targets are rotated into the camera’s identity rotation space by multiplying the camera’s inverse rotation with each of the targets positions, the first task is to calculate their projections.

Please note that in all the figures below (with the exception of the horizontal field of view angle section), the camera is present for reference only, as its final desired position will only be uncovered in the final step.

Targets projections

For each target, four projections are cast to a plane parallel to the view plane, sliced from the camera’s view frustrum. The line described from the target object to its respective projection is parallel to the camera’s view frustrum edges. Relative to the camera, two of these projections run horizontally, and the other two vertically.

Perspective view of the targets projections
Side view of the targets projections
Top view of the targets projections

If any of the target’s projections are outside the camera’s view frustrum (or its sliced plane), then the target object will not be visible. If they are inside, the target object will be visible. This means that the four outermost projections from all targets will define the limit of where the view frustrum must be in order to have all objects in view or partially in view. Adding some padding to these outermost projections (i.e. moving these projections away from the center of the view frustrum plane slice), will result in additional padding between the target object and the camera’s view borders.

Vertical projections

For all vertical projections positions, we are interested in finding their Y component. In the figure below, notice the right triangle with one vertex on the target object and another one on the projection. If we discover the length of the side running parallel the projection plane, that value can be added to the Y component of the target’s position, resulting in the Y component for the upper projection.

Side view of the elements needed to calculate the vertical projections

is equal to half the camera’s vertical field of view angle (). The vertical field of view angle is provided by the camera’s fieldOfView in degrees, which needs to be converted to radians for our purposes ().

The triangle’s adjacent edge length (relative to ) is known, thus we can find the length of the opposite side of the triangle by using trigonometric ratios.

With this, the upper projection’s Y/Z components can be fully calculated. The bottom projection has the same Z component as the upper one, and its Y component is equal to the target’s Y component minus the calculated opposite triangle edge length.

Horizontal projections

Top view of the elements needed to calculate the horizontal projections

The horizontal projections follow a set of similar set of calculations, difference being that we are now interested in finding the X component (instead of Y), and the horizontal field of view angle is used instead of the vertical one. The horizontal field of view angle and its half value () need some further steps to be computed, which will be detailed in the following section.

Horizontal field of view angle

Consider the following figure, in which represents half the horizontal field of view angle, represents half the vertical field of view angle, the viewport width and the viewport height:

Elements for the calculation of the horizontal field of view angle

Using trigonometric ratios, these two equations can be devised:

Replacing in the first equation with the definition of the second one:

Unity’s camera has an aspect attribute (view canvas width divided by height, i.e. ), with which we can finalize our equation and discover the horizontal field of view angle half value.

Outermost projections

Having all target projections calculated, the four outermost ones are picked:

  • is the projection with the highest X component
  • is the projection with the lowest X component
  • is the projection with the highest Y component
  • is the projection with the lowest Y component
All outermost projections

Calculating the camera position

In the transformed space

The X and Y components of the desired camera position in the transformed space are the midpoints of their respective outermost projections, this is, the midpoint between and is the camera’s X position, and the midpoint between and is the camera’s Y position.

The Z component of the camera position in the transformed space is calculated by backtracking a view frustrum from the the outermost projections to the camera Z component candidate. The furthest Z component from the projection plane will be the chosen, in order for the final camera position to contain all targets within its view.

Elements for adjacentv calculation
Elements for adjacenth calculation

Once again, trigonometric ratios will be used to calculate these Z position candidates.

The highest value between and will be picked for the camera’s Z position component in the transformed space.

Final camera position in the original space

With the camera position calculated in the transformed space, we can now multiply the desired camera rotation with this position, which will provide us with the final desired camera position. The actual camera position is then progressively interpolated towards this desired position, to smooth out the camera movement.

Where to get it

The library is available on GitHub and the Unity Asset Store. An example scene of the library’s usage is included. Feedback is most welcome and I hope this can be useful!

]]> ArXiv Papers: React Native Android App http://lopespm.github.io/apps/2018/03/12/arxiv-papers.html http://lopespm.github.io/apps/2018/03/12/arxiv-papers.html

ArXiv Papers is an open source mobile application to search, download and save arXiv papers. It was developed using a react native / redux framework and is currently available for smartphone and tablet devices. You can get it on Google Play.

Motivation

Often I would find myself downloading the same arXiv paper on my phone because it would get buried in the “downloads” folder under the default arXiv id file name, so instead of sifting through a myriad of already downloaded PDFs I would just revisit the paper’s web page and redownload it.

Organizing each paper neatly into its proper folder with a more discernable filename is quite laborious on a mobile device and the apps I tried out either did too much or too little. I just wanted to quickly search an arXiv article and quickly refer to the ones I downloaded.

This seemed not only a good opportunity to leverage on arXiv’s public API and develop this idea into an application, but also to expand from Android native development and dive into cross platform app development, specifically Android/iOS.

Bootstraping

Cross platform mobile app development frameworks come far and wide, so the plan was to go through them, choose one framework, learn its foundations, and attempt to solidify my understanding of its various concepts, advantages, pains and quirks by using it in this new application.

I went shopping for a suitable framework and eventually settled with react native/redux. It seemed like a good fit for the project and a valuable add to my programming tool belt. React and redux had been around for a while and were quite mature and capable at that point. The react native library did not reach a major version yet, but appeared to be quite solid and had a rich ecosystem, vibrant community and various successful use cases in demanding production apps.

I first built my foundations through courses (Dan Abramov’s react and redux courses are wonderful), examples, lectures, tutorials, documentation and articles. I was pleasantly impressed with the CRNA project’s workflow and how it enables you to develop an entire application without touching a single native implementation piece. However, Android and iOS are two distinct platforms, with different design and UX paradigms, meaning that even unejected applications should be specifically tailored for each platform. With this in mind, and knowing that I would need native integrations to download papers for example, I decided to narrow the scope to Android only, but a potential future iOS version should not be a huge pain.

Design

Final Android smartphone app design, made in Sketch

After a few iterations, the above design seemed like a good compromise between functionality, breadth and ease of use. Most importantly, it catered to the app’s motif: search, download and save arXiv articles.

Many, if not all of the apps I explored, allowed the user to browse through arXiv’s categories. For this application however, it did not seem to be an essential feature. Browsing categories serves discoverability, but I am assuming that whoever uses this app would at least have a rough idea of what she is looking for. In that case, keyword search can fill that use case. Looking for DeepMind’s Playing Atari with Deep Reinforcement Learning? Write “deep atari” and the paper you are looking for appears on the first page.

Bookmark-wise, the assumption is that if you download a paper, then most probably it is something your are interested in, thus the higher the probability for its recall. How about merging the two concepts? You are on the article’s details screen, you download it, read it if you wish, and when you arrive back at the home screen, it will be available for immediate recall.

Implementation

React by itself is a library which could be seen as the view in model-view-controller, whereas react native builds upon it by providing native component rendering, and redux is used for state management. Moreover, I chose to use Redux Observable to handle redux action side effects, Native Base for most of the UI components, Flow for type safety, Detox for end to end tests and Jest for unit and integration tests.

Action Side Effects

Redux Observable does not seem to be a popular choice for action side effects, when compared with redux sagas for example, but I think it offers one of the best solutions in this domain. At its core it uses rxJS, a library for reactive programming, which is a natural fit for the asynchronous nature of redux action side effects. This approach is inherently declarative, provides powerful composition (complex async tasks godsend) and makes it easy to cancel async tasks, a common edge case which is often overlooked and can make the difference between an OK application and a very solid application.

Testing

Application logic was contained inside the action’s handlers, selectors and reducers, so I chose not to test the react components because the unit and integration tests could extract an higher value when applied to the before mentioned triad. Adding to this, ESLint was used for code quality control and catching basic errors, and much of the application was typed using Flow. Redux Observable epics were not type checked, since at the moment there are no meaningful type definitions available.

Integration tests were done for the different application modules (articles, papers, etc), using actions as inputs and selectors as outputs; action side effects were tested separately in their respective unit tests. More unit tests could be made to each of the redux units and further satisfy the traditional test pyramid, but the before mentioned static analysis served as a good foundation and most of the module’s inner code paths were covered by the aforementioned tests, so I opted for a more a pragmatic approach, one which could yield a higher value with minimum developer effort.

To encompass the whole application, end to end tests were built with the aid of Detox. Detox’s android support is still nearing completion, but appears to be a very promising solution. One of its most distinctive features, when compared with popular black box testing frameworks like Appium, is its direct integration with the native layers via EarlGrey for iOS and Espresso for Android, translating to more stable, faster and less flaky tests. Using it was straightforward, the API is well designed and it did not stand in the way while writing the tests. Overall, a good experience for this small application.

Folder Structure

Regarding folder structure, I opted for a feature based tree which branched into framework concepts (reducer, action, selector, react component, redux observable epic). As it typically happens in other domains, using the framework’s concepts as the root of your folder structure works great for small applications, but as complexity grows, root division by features makes the structure much easier to extend and reason about.

├── __e2e__
├── modules
│   ├── articles
│   ├── connection
│   ├── donations
│   ├── navigation
│   ├── papers
│   │   ├── __tests__
│   │   ├── epic
│   │   ├── paper.js
│   │   ├── actions.js
│   │   ├── reducer.js
│   │   └── selector.js
│   ├── permission
│   ├── shared
│   └── snackbar
├── configureStore
├── theme
├── App.js
├── epic.js
├── reducer.js
└── selector.js


Get it, Hack it

Feel free to hack away the code or download the application on Google Play, let me know your thoughts about it!

]]> React Native Offscreen Toolbar http://lopespm.github.io/libraries/2018/01/25/react-native-offscreen-toolbar.html http://lopespm.github.io/libraries/2018/01/25/react-native-offscreen-toolbar.html In material design, there is a common scrolling technique in which the toolbar is smoothly tucked away while scrolling down and is made visible again when scrolling up. This behaviour is fairly straightforward to implement when developing a native android app, but for a react native app, the best solution I found was Janic Duplessis’.

The library I am releasing today is an encapsulation of this behaviour and its implementation is heavily based on Janic Duplessis’ approach.

Library usage in both the example bundled with the library and in the search screen of a to be released application

How to Use it

Installation:

$ $ npm install react-native-offscreen-toolbar --save


Usage (full example here):

import OffscreenToolbar from 'react-native-offscreen-toolbar';
 
export default class YourComponent extends React.Component {
    render() {
        const toolbar = () => (<ToolbarAndroid title={'Title'} />);
        const listItem = ({item}) => <Text>{item.key}</Text>;
        const scrollable = () => (<FlatList data={DUMMY_DATA} renderItem={listItem}/>);
        return (
            <View style={styles.container}>
                <OffscreenToolbar
                    toolbar={toolbar}
                    scrollable={scrollable} />
            </View>
        );
    }
}


You can pass any component you desire as a toolbar, but typically this will be a toolbar/navbar backed by react native or a UI library of your choice, like Native Base.

The scrollable however, will need to be scrollable component such as a ScrollView, ListView, FlatList, etc. The OffscreenToolbar will then create hooks around this component in order to gauge the user’s scrolling behaviour and change toolbar’s animation accordingly.

You can also provide the following properties to the component:

  • scrollableOverlay: for a search screen as the one presented above, this component can be used to present an overlay between the scrollable and the toolbar
  • toolbarHeight: adjust this property according to your toolbar height. This value is used for the toolbar’s animation
  • scrollablePaddingTop: since the scrollable is laying behind the toolbar, this property comes in handy to place the scrollable content below the toolbar

Where to get the library

]]> Deep Reinforcement Learning: Playing a Racing Game http://lopespm.github.io/machine_learning/2016/10/06/deep-reinforcement-learning-racing-game.html http://lopespm.github.io/machine_learning/2016/10/06/deep-reinforcement-learning-racing-game.html

Agent playing Out Run, session 201609171218_175eps
No time limit, no traffic, 2X time lapse

Above is the built deep Q-network (DQN) agent playing Out Run, trained for a total of 1.8 million frames on a Amazon Web Services g2.2xlarge (GPU enabled) instance. The agent was built using python and tensorflow. The Out Run game emulator is a modified version of Cannonball. All source code for this project is available on GitHub.

The agent learnt how to play by being rewarded for high speeds and penalized for crashing or going off road. It fetched the game’s screens, car speed, number of off-road wheels and collision state from the emulator and issued actions to it such as pressing the left, right, accelerate or brake virtual button.

Agent trainer implements the deep Q-learning algorithm used by Google’s DeepMind Team to play various Atari 2600 games. It uses a reward function and hyperparameters that fit best for Out Run, but could potentially be used to play other games or solve other problems.

There is a wealth of good information about this reinforcement learning algorithm, but I found some topics difficult to grasp or contextualize solely from the information available online. I will attempt to add my humble contribution by tackling these and also provide details about the project’s implementation, results and how it can be used/modified/deployed.

Let’s start by one of its main gears: Q-learning

Concepts

Q-learning

At the heart of deep Q-learning lies Q-learning, a popular and effective model-free algorithm for learning from delayed reinforcement.

Jacob Schrum has made available a terse and accessible explanation which takes around 45 minutes to watch and serves as a great starting point for the paragraphs below. Let’s take the canonical reinforcement learning example presented by Jacob (grid world):

To implement this algorithm, we need to build the Q-function (one of the forms of the Bell-Equation) by using the Q-value iteration update:

In the above grid, there are 9 actionable states, 2 terminal states and 4 possible actions (left, right, up, down), resulting in 36 (9 actionable states x 4 possible actions) Q-values.

This project aims to train an agent to play Out Run via its game screens, so for the sake of argument, let´s consider that each game screen is transformed into a 80x80 greyscale image (each pixel value ranging from 0 to 255), and that each transformed image represents a state. 6400 pixels (80x80) and 256 possible values per pixel translate to 2566400 possible states. This value alone is a good indicator of how inflated the number of possible Q-values will be.

Multiplying 9 possible actions by 2566400 possible states results in 2566400 x 9 possible Q-values. If we use multiple and/or colored images for state representation, then this value will be even higher. Quite unwieldy if we want to store these values in a table or similar structure.

Enter deep neural networks

Deep neural networks work quite well for inferring the mapping implied by data, giving them the ability to predict an approximated output from an input that they never saw before. No longer do we need to store all state/action pair’s Q-values, we can now model these mappings in a more general, less redundant way. These networks also automatically learn a set of internal features which are useful in complex non-linear mapping domains, such as image processing, releasing us from laborious feature handcrafting tasks.

This is perfect. We can now use a deep neural network to approximate the Q-function: the network would accept a state/action combination as input and would output the corresponding Q-value. Training-wise, we would need the network’s Q-value output for a given state/action combo (obtained through a forward pass) and the target Q-value, which is calculated through the expression: . With these two values, we can perform a gradient step on the squared difference between the target Q-value and the network’s output.

This is perfect, but there is still room for improvement. Imagine we have 5 possible actions for any given state: when calculating the target Q-value, to get the optimal future value estimate (consequent state’s maximum Q-value) we need to ask (forward pass) our neural network for a Q-value 5 times per learning step.

Another approach (used in DeepMind’s network) would be to feed in the game’s screens and have the network output the Q-value for each possible action. This way, a single forward pass would output all the Q-values for a given state, translating into one forward pass per optimal future value estimate.

Image courtesy of Tambet Matiisen’s Demystifying Deep Reinforcement Learning - Left: Naive formulation of deep Q-network. Right: More optimized architecture of deep Q-network, used in DeepMind papers.

Q-learning and deep neural networks are the center pieces of a deep Q-network reinforcement learning agent and I think that by understanding them and how they fit together, it can be easier to picture how the algorithm works as a whole.

Implementation

Above is an overall representation of how the different components relate during a play evaluation, centered around the deep Q-network for playing1, the main decision component.

Each game screen is resized to a desaturated 80x80 pixels image (opposed to 84x84 on DeepMind’s papers), and if you might be wondering why each state is a sequence of four game screens instead of one, that is because the agent’s history is used for better motion perception. Achieving this requires a sequence of preprocessed images to be stacked in channels (like you would stack RGB channels on a colored image) and fed to the network. Note that RGB channels and agent history could be used simultaneously for state representation. For example, with three channels per (RGB) image and an agent history length of four, the network would be fed twelve channels per input state.

The network’s architecture is essentially the same used by DeepMind, except for the first convolutional neural network’s input (80x80x4 instead of 84x84x4, to account for the different input sizes) and the linear layer’s output (9 instead of 18, to account for the different number of actions available)

The algorithm used to train this network is well described here (page 7) and here, but I would like to present it graphically, to hopefully provide some further intuition.

Below is agent trainer´s implementation of the aforementioned algorithm. It adds some new concepts which were not approached by this article:

  • Experience replay mechanism sported by replay memories: randomly samples previous transitions, thereby smoothing the training distribution over many past behaviors
  • Separate training network, cloned at fixed intervals to the target playing network, making the algorithm more stable when compared with standard online Q-learning
  • ε-greedy policy to balance exploitation/exploration

Reward function

The reward function’s definition is crucial for good learning performance and determines the goal in a reinforcement learning problem. April Yu et al. have an interesting paper on simulated autonomous vehicle control which details a DQN agent used to drive a game that strongly resembles Out Run (JavaScript Racer). Based on their reward function experiments, I’ve built a function which rewards logarithmically based on speed and penalizes when the car is off-road, crashed or stopped.

Reward values for when the car is not crashed or off-road

Deployment

Run the trainer and emulator on your local machine by following the guide available on agent-trainer’s readme.

It is also possible to deploy the agent to an AWS EC2 instance or generic Linux remote machine by using a set of bash scripts offered by agent-trainer-deployer.

AWS EC2

Amazon allows you to bid on spare EC2 computing capacity via spot instances. These can cost a fraction of on-demand ones, and for this reason were chosen as the prime method for training in this project, leading to the need for mid-training instance termination resilience.

To accommodate this scenario, the deployment scripts and agent-trainer are designed to support train session resumes. To persist results and decrease boot up time between sessions, a long-lived EBS volume is attached to the live instance. The volume contains the training results, agent-trainer´s source code, cannonball’s source code, dockerfiles and their respective docker images.

Relationship between components when deploying agent-trainer to an AWS EC2 instance

Results

The hyperparameters used on all sessions mimic the ones used on DeepMind’s Human-Level Control through Deep Reinforcement Learning paper, except for the number of frames skipped between actions, which are spaced apart by 450ms (equivalent to 13 frames) on agent-trainer.

The Out Run game, as you would play it in an arcade, clutters the road with various cars in order to make the game more challenging. In-game traffic was disabled for both training and evaluation plays, rendering a more achievable starting point for these experiments. Training with random traffic could be an interesting posterior experiment.

Some experiments were made by increasing the discount factor up its final value during training, as proposed on “How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies”, but did not achieve better stability or rewards when compared to a fixed 0.99 discount factor. The aforementioned paper also proposes decaying the learning rate during training, which increased stability and performance significantly. Decaying the learning rate without minimum value clipping yielded the best results.

Another improvement was to train the game without a time limit, meaning that the training episode would only finish when the car reached the last stage’s end. This allowed for a broader replay memory training set, since the agent traversed a wide range of different tracks and settings.

Play evaluation was the same between all experiments, this is, the agent was evaluated by playing on the default 80 second, easy mode.

Here is a summary of the most relevant training sessions (you can find their models, metrics and visualizations on agent-trainer-results):

Session M Training
game mode		 Learning rate decay
201609040550_5010eps a) timed; easy without learning rate decay
201609111241_2700eps b) timed; easy unclipped learning rate decay
201609111241_7300eps b) timed; easy unclipped learning rate decay
201609160922_54eps b) unlimited time without learning rate decay
201609171218_175eps b) unlimited time unclipped learning rate decay

Training sessions summary: session names are formed by <session ID>_<number of episodes trained>
(M)achine used: a) AMD Athlon(tm) II X2 250 Processor @ 3GHz; 2GB RAM DDR3-1333 SDRAM; SSD 500 GB: Samsung 850 EVO (CPU only training); b) AWS EC2 g2.2xlarge (GPU enabled instance), 200 GB General Purpose SSD (GP2)

Agent playing Out Run (timed easy mode, no traffic)
Session 201609111241_2700eps

Agent playing Out Run (timed easy mode, no traffic)
Session 201609171218_175eps

Notice on the videos above how the timed mode trained session 201609111241_2700eps reaches the first checkpoint about five seconds earlier than the unlimited time mode trained session 201609171218_175eps, but proceeds to drive off-road two turns after. Its stability gets increasingly compromised as more episodes are trained, which can be observed by the rampant loss increase before 7300 episodes are reached (201609111241_7300eps):

Training metrics for session 201609111241_7300eps
Play evaluation metrics for session 201609111241_7300eps: using ε=0.0; evaluation made at the end of every 20 training episodes
Training metrics for session 201609171218_175eps
Play evaluation metrics 201609171218_175eps: using ε=0.0; evaluation made at the end of every training episode

Both 201609111241_2700eps and 201609111241_7300eps timed trained sessions mostly drive off-road and stall after the first stage, whereas the unlimited time mode trained session 201609171218_175eps can race through all the stages crashing only three times (as shown on the article’s first video) and is able to match the performance of a timed trained session when evaluated on the default easy timed mode.

Below is the loss plot for 201609160922_54eps and 201609171218_175eps, both trained using the game’s unlimited time mode, difference being that 201609160922_54eps keeps a fixed learning rate and 201609171218_175eps decays it every 50100 steps:

Loss comparison between sessions ■ 201609160922_54eps and ■ 201609171218_175eps, as viewed on tensorboard

Other representative visualizations:

t-SNE visualization, generated by letting the agent play one game on timed easy mode. Agent is using the network trained on session 201609171218_175eps
t-SNE visualization, generated by letting the agent play one game on unlimited time mode. Agent is using the network trained on session 201609171218_175eps
Visualization of the first convolutional network layer’s filters. These can be viewed via tensorboard


Final Notes

Plugging other problems and games

Agent-trainer was not built from the get-go to train games or problems other than Out Run, but I think it would be interesting to perform a thought exercise on what would be necessary to do so.

There are three main areas in which agent-trainer has domain knowledge about Out Run:

  • game package, which contains
    • Action enumeration: describes all the possible actions in the game.
    • cannonball_wrapper module: only this module has access to the cannonball emulator. It translates the aforementioned actions into game actions and is accessed by methods such as start_game(), reset() and speed().
  • RewardCalculator class: contains the reward function. Instead of using a generic reward function like DeepMind, it was chosen to have a tailor-made reward function for Out Run, which takes into account the car’s speed and its off-road and crash status.
  • metrics module: aware of the speed metric, which is Out Run specific, and score, which is game specific domain knowledge.

Training another game would require the creation of a new wrapper with the same interface as cannonball_wrapper, a new Action enumerator specific to the game, a new RewardCalculator with a different reward function and the removal/replacement of the speed metric.

Apart from the previously mentioned steps, solving generic problems would require the preprocessor to be changed/replaced if images were not to be used for state representation. An option would be to create a new preprocessor class with a process(input) method, tweak the hyperparameters as required (so that the network knows which dimensions to expect on its input), and finally inject the newly created class in EpisodeRunner, replacing the old preprocessor class.

Further references

I am not a machine learning expert, but from my learner’s point of view, if you are interested in getting your feet wet, Andrew Ng’s Machine Learning Course is as a great starting point. It is freely available on the Coursera online learning platform. This was my first solid contact with the subject and served as a major stepping stone for related topics such as Reinforcement Learning.

Udacity Google Deep Learning: this free course tackles some of the popular deep learning techniques, all the while using tensorflow. I did this right after Andrew Ng’s course and found it to leave the student with less support during lessons - less hand-holding if you will - and as result I spent a good amount of time dabbling to reach a solution for the assignments.

As a side note, I started building this project by the end of the Deep Learning course, mostly because I wanted to apply and consolidate the concepts I learnt into something more practical and to share this knowledge further, so it could hopefully help more people who are interested in this.

Other useful resources:

Source code

All source code is available on GitHub:

  • Agent Trainer: the core python+tensorflow application
  • Cannonball: custom Cannonball (Out Run game emulator) fork which contains the changes needed to access the emulator externally
  • Agent Trainer Deployer: bash scripts to deploy agent-trainer to a generic remote machine or AWS EC2 instance
  • Agent Trainer Docker: Dockerfiles used when deploying agent-trainer to a remote machine
  • Agent Trainer Results: Collection of training sessions, each containing their resulting network, metrics and visualizations


  1. “Deep Q-network for playing” in this project is equivalent to DeepMind’s “target network $\hat Q$” and “Deep Q-network for training” is equivalent to DeepMind’s “network Q”↩

]]> Compressed Deque in Python http://lopespm.github.io/libraries/2016/08/27/compressed-deque.html http://lopespm.github.io/libraries/2016/08/27/compressed-deque.html This post aims to give some insights about a recently open-sourced python deque which compresses its items for a decreased volatile and persistent memory footprint.

Why? How can it be useful?

Compressed Deque came up while buiding an algorithm which relied on fast random accesses of numerous complex objects. When compressed, this blob of data could easily fit in fast volatile memory, as opposed to being fetched from a slow hard drive. Even though there was a performance penalty from compressing/decompressing items from volatile memory, this solution proved to be faster than accessing these items from indexed files on a hard disk (on a SSD the gains are more negligible though).

This collection might also be useful for programs running on devices with very limited memory available.

Structure

Compressed Deque inherits from deque and stores its items as zlib compressed pickles. The middle pickle layer only serves as a generic serialization method which can provide a serialized object string for zlib to compress. Although pickle can compress objects, its compression rate does not match zlib’s, even using higher protocols.

image

save_to_file() and load_from_file() static methods are provided to persist the collection directly into disk in its compressed representation, without much overhead.

The persisted file contains a sequence of header/compressed_value pairs: the header is a 4 byte integer description of the compressed value’s length and the compressed value is similiar to its in-memory representation.

image

How to Use it

An easy way to install this package is by using pip:

$ pip install compressed-deque


You can then import the class and use it:

from compdeque import CompressedDeque

# Instantiate the Deque
collection = CompressedDeque()

# Use it as a normal deque
collection.append(1)

# Persist to a file
CompressedDeque.save_to_file(collection, file_path="/path/to/collection.dat")

# ...

# and load it when you need it later
loaded_collection = CompressedDeque.load_from_file("/path/to/collection.dat")


Source Code

Source code is available here: https://github.com/lopespm/compressed-deque

]]> Attenuate GoPro Chest Mount Heartbeat - using GarageBand/iMovie http://lopespm.github.io/audio/2015/12/08/gopro-heartbeat.html http://lopespm.github.io/audio/2015/12/08/gopro-heartbeat.html If you have a chest mount (chesty) for your GoPro, then you might have had the surprise of hearing your heartbeat on the recording, since it may lay fairly close to your heart, picking up its beat. This post will:

  • show you how to attenuate the heartbeat sound to a point where it is almost imperceptible, while maintaining the overall sound
  • attenuate the difference between quiet and loud sounds, so the listener gets less startled by a sudden noise created when entering with the bike on rought terrain or the common bike bell sound.

Steps, using GarageBand/iMovie

In this specific case, GarageBand 10.0.3 and iMovie 10.0.5 were used, but other versions should work just as fine.

  1. Open iMovie
    1. Import your GoPro file(s) into your iMovie project. This is the best point in time to edit them and arrange the movie to your liking
    2. Export your movie (File->Share->File…)
      • Open GarageBand
    3. Drag the exported movie into your GarageBand project
      • Decompress gopro_master_track.patch.zip and copy gopro_master_track.patch to the “~/Music/Audio Music Apps/Patches/Output”
      • Go to your Master Track (if you do not see it, click “Track->Show MasterTrack”) and click “User Patches” in your Library tab (if you do not see it, click “View->Show Library”)
      • You should now see inside the “User Patches” the patch file we just copied, select it
      • Your audio should now have an attenuated heartbeat and a more audible and balanced sound
      • Export the audio (Share->Export to Disk). AIFF or MP3 will work great. I personally use the MP3 192kBit/s export, since the GoPro audio is not extra pristine to start with, so the tiny losses in compression are pretty much neglegible
      • Go back to iMovie
    4. Remove the sound on your original GoPro video(s)
      • Drag the export audio from GarageBand, and place it below the video timeline
      • Done!

Why did this *hopefully* worked for you?

The whole concept is very simple really, the heartbeat is simply being attenuated on specific frequencies by a Channel EQ plugin included in the above Master Track patch. This is how it looks visually:

image

This Channel EQ alone will turn this:

into this:

And that is it! The rest of the plugins included on the Master Track Patch are:

  • An additional Channel EQ that attenuates clicking sounds I heard along my videos
  • Two compressors and a limiter which increase the overall loudness and diminish the difference between quiet and loud sounds

Support Files:

  • Master Track Patch (which includes all the above plugins)
  • Channel EQ preset to attenuate the heartbeat sound. This can be used by opening a Channel EQ plugin in any of your tracks and clicking “More->Load..” inside it
  • Example video in which these plugins were applied


]]> Quickly Customize OSX Terminal Style and Functionality http://lopespm.github.io/workflow/2015/01/27/customizing-terminal.html http://lopespm.github.io/workflow/2015/01/27/customizing-terminal.html

Style - how it looks

Add the real world tested Solarized (dark/light) Terminal profiles: https://github.com/tomislav/osx-terminal.app-colors-solarized

  • After cloning the repo, just open the profile files, which will add them to your Terminal.app preferences
  • You can set them as the default profiles on your Terminal.app preferences
  • To enable the terminal colors, add the line export CLICOLOR=1 to your .bash_profile

Function - what it does

Grab the .bash_profile here: https://gist.github.com/lopespm/3877ff1ca4064a392686, which is heavily based on Nathaniel Landau’s profile

  • In OSX, .bash_profile is executed everytime you open a new Terminal window, and it lives in your user’s home directory, i.e. ~/.bash_profile
  • If it does not exist, just touch ~/.bash_profile to create it
  • If you do not want to clutter your .bash_profile, you can create a file like ~/scripts/utilities.bash with the portions you find useful and then add this line to your .bash_profile: source ~/scripts/utilities.bash

Hopefully now your Terminal is even more inviting :)

]]> Survival Ball on OUYA http://lopespm.github.io/games/2013/09/08/survival-ball-ouya.html http://lopespm.github.io/games/2013/09/08/survival-ball-ouya.html Survival Ball in now available on OUYA, and its publishing process was fairly straightforward. From the day of submission (2nd of September) to its approval (5th of September), it took a mere 3 days, counting that a re-submission had to be made in-between.

Survival Ball was already available on Google Play, Amazon and Kongregate, but OUYA just felt like the perfect platform for the game. With it, you could just grab the OUYA controller and/or a spare XBox/PS3 controller, and just have a quick match. I really loved the concept, and re-iterated on the previous version of the game to give it more controller friendly menus, better graphics, refined textures, and an overall revamp.

After finishing up the port to OUYA, on the 2nd of September, I’ve made the first submission. I’ll just point out that upon this moment, it seemed to be stuck with a red status message which read “Still verifying apk…”. To resolve this, I edited the submission, re-submitted the APK, and waited a few seconds. The game was then submitted for approval.

On the 4th of September, a re-submission was asked by the OUYA review team (which were very friendly by the way), since some elements fell of the safe-frame, which is safe-guard for gamers who have screens that perform too much overscan.

The game was re-submitted on the same day, and was approved on the 5th of September. From this point on, one can release the game whenever it seems more appropriate. The game was released on the same day.

I’ve came to learn during this process, after publishing the game, it first goes to the “Sandbox” section. Only after a major acceptance from the game community, can the game be promoted to the primary categories. So, if you would like to take a quick peek at the game, you can find it at the “Sandbox” section.


]]>