My analysis report or How to analyze large amounts of test data

on Monday, February 28, 2011
The past two weeks and today, I've been continuously working on my analysis report. Today, just before lunch, my first draft (which still needs introduction and conclusions) was rolling out of the printer. I thought it was not so bad how I put together a 23k words document and a large number of graphs and tables in only 11 days' time (including the time to make the analysis calculations).

I wasn't in my best shape in terms of concentration (my housing situation has cause me a lot of tension lately, although the final solution is very near), so I've been randomly clicking around the interwebs too much. However, I still think my workflow with regard to making this document was neatly organized.

Only a few months ago, I spent about the same amount of days on an analysis document of less than half the size of this one, and I spent all those days working until 8pm (instead of 5 - 6pm as I'm trying to do now).

So here's how I organized this project:

1. Review what is already done.

I already had two previous versions of the analysis report of my experimental work, so I could recycle at least some material. I ended up mainly recycling the style of my graphs, and reanalyzing my data since my new test results were screaming to be included in every possible subchapter. I also knew my advisor had suggested changing the amount of data points I'm squeezing into the plots. I waited until this version to make this change.

2. Check the expectations.

Before starting (or better, before realizing another deadline in our project was coming up), I had an appointment with my daily supervisor who told me very clear which questions our funder wants to be answered. I really appreciated that I knew exactly in advance what the expectations for this report are, so I can work towards this in as much as I can.

3. Know what needs to be done.

So I knew what the expectations were. I printed out a previous version of the report and started to reread it. In every section I jotted down what had to be altered, recalculated or added. I also changed the order of the sections, and ended up with a document filled with pencil-scribbling to guide me what had to be done, section by section. I started off with putting the sections in my word document into the right order, and then I could get started altering, improving and adding material to the sections.

4. Give the reader some framework.

Previously, I just referred to the reports with the test data. I now added a short chapter with a sketch of the test setup and some basic information about the specimens. I think that was a good idea.

5. Study the parameters as resulting from the tests.

And so I've spent 11 days playing around with data in Excel and having rows and rows and columns and columns of data flying around before my eyes. My first action was to filter my data into nice plots and tables per parameter which we had been testing.

6. Compare to calculation methods in general.

My second action then was to compare the experimental values to the calculated values. Turn out, I had to recalculate most of my calculated values as I had only made rough predictions before the tests, and I did not implement the properties as we had measured them on the test data. Luckily, I have a nice set of MathCad sheets that do the job for me. Also, once I had a spreadsheet set up to analyze my data through one method, using the next method was only a matter of copy and paste.

7. Compare how the studied parameters are reflected by these methods.

Since the comparison with the Dutch code as calculation method gave me quite disappointing results, I decided to go and check per parameter the comparison of test value to calculated value to see where exactly the weak spots of the methods are. Doing that, I did some nice observations, which motivated me to play around some more with all my data.

8. Give recommendations.

It doesn't really help the fund to produce graphs and tables and let them look for the answers to their questions in all the material I've produced. So today, I printed out the document, started reading from the beginning, and jotted down the most important recommendations - per calculation method and in general.

9. Review your material.

Does it make sense? When reviewing my material today, I found that one of my tables raised my eyebrows. I went back to check it, doublechecked my calculations, and then I discovered that I had been reading results from the wrong column.

How do you analyze large amounts of test data? Any advice for me? I'm just hoping to give my funder a report which gives them as clear as possible the answers to their questions, but also gives them enough background to these answers.

Reimbursement

on Sunday, February 27, 2011
I was happy to see that there is a new PhD comic today, titled "Reimbursement"

I've heard similar stories, of course, but I am glad to write here that TU Delft has a perfect method for travel arrangements, and I think it could inspire other universities as well. Traveling for conferences can be expensive, and for graduate students it can be financially very painful to be waiting and waiting until after the conference the reimbursement can be arranged. I really appreciate the way my university has solved this, in what I think is a very elegant way.

When I plan to go to a conference, it is important to start arranging well ahead of the registration deadline. Here's how it works:

Registration
I fill out the registration, but the central finance services of university take care of the payment for the registration. It does take 4 to 6 weeks to arrange it, but I don't need to pull out my credit card at all.
WIN!

Flight
I contact the university's travel agency, and they book the flight for me and e-mail me the link to the e-ticket. There's also a group insurance to take care of us in case something goes wrong.
Again, this doesn't cost me a cent at all. WIN!

Other costs
The secretaries have files in which examples of estimated costs per major city in all possible countries are listed. Based on those rates, an estimate of our costs during our traveling is estimated, and two weeks before the conference 70% of these costs are put into our account.

I think this is a very good solution, and a great way to stimulate graduate students to get moving and take their story abroad without the menace of an excessive credit card debt.

Strengths or Challenge?

on Saturday, February 26, 2011
After reading (amongst others) this article about using your strengths to find your most suited path, I've spent quite some time reflecting on this issue.

The fact is, I've never made a choice based on my strengths. In fact, I've always been driven by the challenge. I've never taken the option which I thought was best suited for me, but I've always taken the most challenging option.

In high school, my favorite subjects were Latin and History. Classical studies or history would have been a logical choice of study for me, but I chose engineering because I thought it was going to be a though study, and I felt as if I needed to look for a challenge to open up my maximum potential.

Whenever I feel like I've reached the point at which I am comfortable at doing what I am doing, and I feel like I'm mastering the task or subject, I start feeling a bit restless. I feel the need to go one level up and challenge myself again.

Reading about building your life path based on your strengths sounds very logical to me, but I wonder if it doesn't deprive us of some options. The freedom to learn from your mistakes, to play around and see if we can grow in a skill which initially does not seem to be our best fit.

I've come to think that not all of it is as contradictory as it would seem at first. I now think that knowing our strengths, cherishing and nourishing them is indeed something which will make us more confident and will let us live at our full potential. Within that frame, there is still room for improving ourselves through trial and error, as long as we keep our strengths and core values in mind. I've ended up realizing that one of my main strengths is my passion for challenges.

Magritte

on Thursday, February 24, 2011
Last Sunday, I -finally- visited the Magritte museum in Brussels. I would highly recommend everyone to visit this relatively young (open since 2009) museum. Details can be found here.

Although I grew up with an admiration for Magritte, I didn't know anything about his two "other" styles: besides his surrealist works, he also painted some impressionist works (during the second world war) and in a "periode vache" (after the second world war).

Impressionist work by Magritte

Work from Magritte's "periode vache"

These works were not that popular, and Magritte's gallery holder in Paris told him to return to his old style. That must have been the best advice someone ever gave to the painter. He returned to his surrealist style, and produced some of my favorite paintings.

Magritte challenges our perception of day and night
I truly love how Magritte's paintings challenge our perception of reality. They make me think and reflect on our world.

At the exhibition, booklets with citations of Magritte (which are displayed in French on the walls) are available. As food for thought, here are five citations of Magritte:

All that I desire is to be enriched by intensely exciting new thoughts.

The real value of art is measured by its capacity for liberating revelation.

Surrealism is the immediate knowledge of reality.

We mustn't fear sunlight just because it almost always serves to illuminate a miserable world.

To be surrealist is to banish the notion of 'deja-vu' and seek out the not yet seen.

Seven simple steps to tackle writer's block

on Monday, February 21, 2011
As a PhD student, writing is one of my main activities. I've been trying to improve my academic writing over the past year as part of my PhD journey. Although I haven't had to face a giant writer's block yet, I do remember staring at a white screen for hours and hours, trying to write an abstract, but always deleting parts and trying to start over new. With a few basic principles, I've overcome this slight panic which I feel when I'm faced with a new document, staring at me in all its whiteness.

1. Don't postpone writing 

Don't wait until you're in your last year and need to start writing your thesis. Although this statement is common knowledge by now, there are still students who wait until all their experiments are finished to look at their data, analyze them and then write about it. By the time these students reach the point of writing, there is quite a burden on their shoulders. Students like me, who are funded from outside university, need to write partially finished reports whenever a deadline of a part of the project is near. This pushed me to start off with the "easy" writing: the description of every single test which I've been carrying out. An introduction to the test setup and measurement methods came along with that, and without noticing I had some basic material to describe my experimental work.

2. Learn from your examples

There's nothing wrong with being inspired by a great paper or a very clear thesis. The outline of my papers is usually tied to a strict scheme since I have been mainly describing my experiments, but I've been inspired by other papers that describe their experimental program very clearly. When someone has a good and structured setting out there, it's an ideal opportunity to learn from it and use it as a basis for your own writing. Your final product won't look anything like the original.

3. Have material ready

I have probably more (not-so) random documents on my computer with loose ideas, citations and copy-pasted pictures and paragraphs from papers than I have finished documents. I love having material ready and being able to start brewing a story from some ingredients. It feels much more motivating to me than to be staring an empty document.

4. Body first

Don't bother finding the perfect opening sentence for your paper. Just start with the core of your writing: some background information and literature review, and then off to your own contributions. Once these thoughts are out there, it's much easier to write an introduction and conclusion which embody the main ideas of your piece.

5. Let the ideas flow

Try not to correct yourself and reread too much while you are typing out your ideas, as this will only slow you down. Emptying your head first helps to let the ideas flow out in your writing. You'll be proofreading anyway later on, so you'll still have plenty of opportunity to correct and improve every sentence. However, having an entire part written, will help you keep an eye on what really matters: the flow of the story, not a collection of perfectly written sentences. 

6. Explore different facets of writing

I've always quite liked writing, but I used to keep a distinct line between my creative writing and my university stuff (which I didn't enjoy writing previously). Over the past few months, I've exploring as many facets of writing as possible: writing poems, writing my diary, keeping this blog, trying to force my thoughts into 140 characters (and I hope to add writing short stories to this list soon as well). Having the habit of writing helps all possible forms of writing, academic writing included. Writing can be lots of fun once you've learned to appreciate it.

7. Fresh air

Still staring at a blank sheet,  no matter what you've tried? Then don't be too harsh on yourself and go and get some fresh air. Play outside, have some fun, sleep over it. Tomorrow gives you another possibility (assuming you've been good at planning and you're not writing a few hours before your deadline).

What helps you to avoid staring at an empty document and gets you started to write?

Five ways to jumpstart your day

on Thursday, February 17, 2011
For a PhD student, I am quite an early bird. I start every day at 8am, and I arrive to my office some time between 7:50am and 8am.Having a steady routine seems to be a great way to work continuously towards my goals.

Every now and then I start the day (too) slowly, and I end up without much work done before noon. To avoid this, I have some ways which help me to get my day started right away.

1. Don't snooze

I've never really understood why people like to push the snooze button on their alarm clock anyway. Once I am awake, I just come out of bed and start my day. Before I even realize that it's rather early, I'm already washed and dressed. 

2. Breakfast!

I'm not a nutritionist or anything, but you can't ignore all these articles which say that you need a good breakfast to start your day. I just always wake up very hungry, so skipping breakfast is not an option for me. What I've noticed over the years, is the importance of my breakfast. I've shifted from sugary cereal to honey covered cereal to wholesome cereal with dried fruits and nuts. I started to notice how much more energy I get from a breakfast without all the sugar. Recently, I've also started to have oats in the morning. The hot milk makes me a bit sleepy, but the oats themselves fuel me for a long morning.

3. Bike to university

Nothing wakes me up as well as a good bit of exercise. Biking to university involves climbing uphill (on a bridge, that is), and that makes my heart all pumping and ready for the day. I like clear, crisp mornings which give me a good dose of fresh air. I really dislike the foggy, smoggy days on which I feel I can hardly breathe.

4. Know your most important task for the day

I usually leave in the evening my most important task for the next day on my otherwise cleared desk. I also keep the document I need to work on opened on my computer. That makes me ready to get started with it right away.

5. Enjoy the silence  

(not just the Depeche Mode song). I can get great progress in the morning before people start walking up and down the hallway and students start dropping by to ask questions. My officemate is an early bird too (he starts at 7am), but his presence doesn't disturb me as much as the footsteps in the hallway. I assume the footsteps distract me so much because I always try to figure out if they are on their way to my office to come and ask me something, or are headed somewhere else.

I know this all sounds really good, but too often I start with reading e-mail and surfing the internet. I'm still working on breaking this habit and doing these low-energy tasks at my low-energy moments of the day.
How do you get started in the morning?

5 tips to boost your productivity

on Monday, February 14, 2011
Today has been a day of great and fast progress. At 2pm I had a meeting with my supervisor to discuss a proposal that I had written for a design code which is in progress. We discussed it, and he asked me to add more to the background document I wrote for it. When I asked him how much time I had for it, he informed me that the meeting is this weekend, so that he would need it before Friday.
I calculated in my head that it would probably take me 2 or 3 days, so I should be able to do it. The timing was a bit unfortunate, as I was just informed about another deadline a few days ago.
At 3pm I was back in my office, with a nice warm cup of tea and a clear desk to start on this project. At 5:55pm it landed into my supervisor's mailbox.

So what gave me my productivity wings today?

1. Time pressure

I had a clear deadline, given by the person who has the highest authority over me (although I'm not treated as if there is some hierarchy in our university), and I knew he is taking this material to a very important meeting. I felt very motivated to simply pull out my sword and slay this calculation dragon right away.

2. A clear and defined task 

I knew exactly what I was expected to do, and so I could start right away doing it. Most of the time, my tasks are not very defined. I'm asked to do something rather vague (write an analysis report and pay extra attention to the dutch design practice), so I need to spend quite some time thinking how I will approach the task, set out what I precisely need to do and only after defining these steps I  am able to actually get started.

3. Curiosity

Since this piece of my work is taken to an important meeting, I felt that this is an opportunity for me to show some experts what I am doing (although it's taken to the meeting as a formula and its background document). Other than that, I was simply curious to go and carry out my supervisor's suggestions. The largest tasks he suggest were related to checking the validity of my formula against some test results from the literature. I was quite curious (and a bit anxious) to see if my little mix-and-match formula would be standing strong when comparing it to other experimental work.

4. Having material ready

I could provide such a quick response since I had all the main ingredients for the calculations ready. I already had a little summary of those test results from the literature in the literature review on which I am still working. Besides that, I also had a spreadsheet ready which I used to compare my test data to my newly developed formula. All I needed to do was to alter my spreadsheet so that I could use it for the experiments from the literature, and then plug in the properties and take the answer out of it.

5. An after-lunch walk outside

I was very sharp today, and I think one of the reasons is that I finally went to have a walk outside after lunchtime. I didn't feel my afternoon dip at all today, even though I'm emotionally tired and distracted as a result of all the trouble with my roommate. Before I started tackling the task at hand, I also opened my window for a few minutes to let in some fresh air. I think my office really needed some air traveling through it, since I've been keeping my door closed for most of the time and the windows haven't been open since September.

The importance of good communication

on Sunday, February 13, 2011
I haven't been writing much lately, nor have I been doing my best at meditating, getting enough sleep, focusing and all the other good plans I had for 2011. I've been stressed out and tense for the better part of the past month, with more and more tension until now.

The reason for this is that there was tension between me and one of my roommates. There's no need for me to write out here what went wrong, but I've learned an important lesson from this conflict.
Talking can make you avoid a lot of trouble.

And that is probably one of the main reasons why tension between two people sharing a house has led to a friendship being completely ruined.
I tried twice to have a serious conversation in which we would be able to look for solutions all together, but we barely exchanged any thoughts. And the tension remained.
I tried again last Thursday, after several days of being sick in my stomach, proposing a temporary solution, and I only got a reply of "We might look into that." And then I was ignored and they (roommate and husband) were staring at their laptop screens again.

To their friends and family, of course, they have been portraying me as a horrible person who has done them terribly wrong. However, I really wish they would realize that they could have avoided a lot of problems when they would have shared their ideas with me and would have let us find a good solution. My roommate changed her planning (what we agreed upon before she moved into my apartment), and her current planning is basically non-existent. This already led to a lot of doubts for me, as I had no idea what we would be up to in our house.
Some good communication could have saved all of us from the stress this situation brought us.

How I started to take my own deadlines and planning seriously

on Monday, February 7, 2011
In this post, I'm elaborating on the first way to motivate yourself which I described earlier on. 

The very basis for my habit of taking my own planning seriously dates back to the time when I was studying in Brussels. In the Belgian system, exams only come twice a year, at the end of a (if I remember correctly) a 13 weeks period of classes. When I started, some courses were still year-long courses and if we'd fail the exam after the first semester of that course,  we could still recover the course by taking an oral exam on the entire course at the end of the academic year. After 13 weeks of class, we were having 3 weeks to study and then 3 weeks of exam, with usually 3 days in between the exams. Homework is not coming that often as in the US system, and the amount of theory is typically much larger in my impression. We'd have to memorize entire books with proofs and derivations for every course.

In my very first year I ended up with 50cm of paper which I had to put into my head some time during the next 6 weeks. I was not having class, so I was mainly in isolation, and with an amount of material I  had never seen before. I was not too positive about ever being able to pass any exam at all. And I indeed failed Linear Algebra (I'll never forget that), and I barely passed Chemistry. Before that, I had always passed all exams in school with flying colors, but suddenly I was struggling. I was merely struggling with the amount of material I had to work through, taking my short concentration span into account and my average need for sleep and relatively large need for breaks and creative activity.

In the Belgian system, every class typically leads to two exams: a theoretical exam (about the derivations of formulas or proofs which you should study) and an exam of exercises. I used to do pretty bad on the exercises, since I never really managed to prepare for those. Or I would try to work through a bunch of examples the day before the exam, not giving my brain the time to really master the material.

Bit by bit, I realized I had to change my way of working. One reason was that I wanted to get rid of the insane amount of stress I'd have during those weeks of exams (I'd get sick almost every time) and the other reason was that I wanted to master some subjects - not just study to pass the exam and then forget the material.

So I started to study during the year. With 40 hours of class a week and compulsory labs which need preparation, it was hard to find time to study (and still being able to practice sports, take music lessons, sing in a band, go out and all that). I started to actually understand what we were doing in the exercises, and put all the material together and understand the whole story behind all the theoretical work. I would make a planning half-way the semester and start working through all the proofs and derivations in the books. Some classes I skipped deliberately to work through the material on my own, during the hours of the lecture. I found what was working best for me, but in order to be able to do so, I really had to respect my own planning.

As my planning became better, I started to get better results, started to be respected by my classmates and my self-confidence grew along the way. I also started to enjoy studying. In spring, I would take my plastic table outside and sit in the pleasant spring weather to study during the day, and then I would reward myself with relax-time in the evening.
Getting such positive results from learning to work in advance became my main motivation to start in time and respect my self-imposed deadlines. By now, it has become a habit and I would never want to end up getting so stressed that I am brain-paralyzed again.

Ten tips for a better concentration

on Thursday, February 3, 2011
One of my New Year's resolutions was that I would be working hard towards a better concentration and focus. My attention span is extremely short, and I have never been able to stretch it longer. I am not sure if this is because my brain just functions like that, or if it is because I could always get my schoolwork done in short amounts of time, and never got used to concentrate for longer than 20 minutes.

I've started to find what works for me, and thought it was time to track my process, and what I've learned from it so far. Here is a list of 10 insights I gained while trying to work on my ability to focus.

1. Set smart goals

If I want to concentrate, I need to know precisely what I am concentrating on, and how much time I plan to spend on it. If not, I find it too easy to drift off. If I tell myself that I am now taking the entire month to get some serious progress on my literature review,  I am very tempted to let my mind float around. It helps to tell myself that I now am going to read this paper in this much of time.

Smart goals are a great way to stay realistic about your progress and your planning. Among other explanations, SMART stands for:
Specific
Measurable
Achievable
Realistic
Time-bound

2. Tackle procrastination 

Make today your no-procrastination day. Just say goodbye to it for once, and feel great about all the progress you can do. Imagine you are working towards a deadline. 

If you really feel that you can't beat procrastination, then start keeping a little log. Write down when you went off the path, started procrastination, how you were feeling. You might start to see a deeper pattern (Do you procrastinate when you're hungry? Does it happen after lunch because you are too stuffed? When you haven't had enough sleep?). Just take the decision for yourself to change it. And then pat yourself in the back for every time you felt like going to wander off but stayed on track.

3. Know your "brain rhythm"

 I've acknowledged for myself that I need to work in chunks of 20 minutes of concentration at a time. Even though I am trying to stretch this extremely short time span, I am now accepting this as the wave of my brain that I need to surf.
Just getting to know within which time I can concentrate already helped me to set that time as a fixed span of time to concentrated. After 20 minutes, I allow myself to look around and let my mind vent all its busy thoughts, but then I try to block them out again and focus for another 20 minutes.

4. Know your "body rhythm"

I've come to understand that I work best when I keep regular hours in my office, start at 8 am every day and go to sleep at 11 pm. I'm at my sharpest in the early morning, then I slack off towards lunchtime, and then I get my second peak around 3pm, until I go home around -typically- 6 pm. I usually have another peak in the evening, after coming from the gym, which I either use for additional reading or non-phd creativity bursts. Since I've discovered this (and I don't think caffeine can do much about this), I've been trying to place my heavy concentration activities during my peak moments, and wait for phone calls, e-mailing and the like for the moments of the day when my body is not at its fullest.

5. Clean desk

If you have a messy desk, all you need to do is to clear it out once. At the end of every day, I sort out what is left on my desk, and in the morning I can start on a completely fresh (or with my most important task for the day read) desk.
As a student, I used to work on the messiest desk you can imagine, and I even had piles with random papers and books around my desk on the floor. At some moments I couldn't even reach my bed without having to jump over the stashes of notes and homeworks. It took me so much energy to keep track of what was traveling around where, so one day I decided to change it.
It now really helps me to have only what I am working on at my desk, and not all possible shiny tings I can start playing around with.

6. Eliminate distractions 

What breaks your concentration? For me it is a wide variety of elements: noise and other sensory impulses, wandering of my thoughts and -when I am working on my computer- the temptation to go and surf around the internet.
Once you know what distracts you, you can try to eliminate it. For me, that means that I've been using a blocksite tool for a while to block the website of the news and facebook from my browser. I've now disabled it, because I got used not to surf around that much anymore (although, I admit, I've been slacking off this week again). Eliminating distractions also means that I need to close my office door regularly, if not I just start following all conversations in the hallway.

7. Meditate

I'm still fully experimenting with all this, as you can read here, here, here; here; here, here, here and here(looks like I've been writing quite some lines about my efforts and good intentions already, more than I realized).
I am convinced though that I can train my mind with this, and that it will result in a better focus.

8. Sleep enough

You can't concentrate when you're sleep-deprived. Even though our entire society is in a constant zombie-mode, it is important to discover how much sleep you need and make sure you get it.

9. Take time to unwind

I've discovered that I really need to relax on a daily basis in order to keep up with my PhD progress. I benefit a lot from going to the gym and sweat it all out, and from my choir practices. Sometimes I feel guilty about taking time for myself, while I could be reading another paper, but I've started to understand that for the quality of the hours I spend in my office, it is important to relax and get the most out of those hours.

10. Don't get frustrated

Whenever you do start drifting off, or can't concentrate, don't blame yourself (provided that you did an effort to focus). I've had a week with a lot of tension because of the situation with my housemates, and I got all upset because I wasn't making progress on my literature review. It only made it worse. I've decided to let the pressure loose for a while, and that helped me relax, take a deep breath, and get ready to concentrate again.

What are your ways to find a better focus? Please share them with me, I'm still trying out how to concentrate better...