Migrating WordPress Hosting Provider to Google Cloud Platform

Introduction

Hosting is one of those things that you tend to buy and then forget about, like buying an electricity provider. It just kinda works and you forget about it. There are always specific cases where you may need someone more specialised, but for the most part, it just works. But every now and then, you there comes an event where you decide that a change is in order. For me, I started working for a cloud provider AND my current hosting provider was up for renewal (and they were expensive).

Uh oh. Bad combination right there! Here’s my high level wordpress migration script.

Note, this takes a couple of weeks to execute, including the domain (DNS) transfer, so keep this in mind. I finished my migration with about a week to spare. Whew!

Caveat Emptor: Warnings before you commence on this quest

  • This is my personal blog and opinion and doesn’t represent the opinions of my employer nor a detailed study into your requirements. Take this into account when you read my instructions here.
  • Following this deployment guide will make you responsible for your WordPress instance. This assumes that you have the capability to manage it (i.e. support it from an infrastructure and code perspective). If you do not, this may not be for you (however you can try it bar the final service cutover and shutdown of old service for training purposes).

Step One: Preparation for migration and check your requirements and  budget

  • Select your future cloud provider (in my case, Google Cloud Platform)!
    • Analyse your current cloud provider / service provider and work out what you require in terms of equivalent compute, storage, network, and support.
    • Going fully managed if support is really important: If you do not want to rely on community support and/or are not comfortable with development / debugging a LAMP stack application, perhaps GCP is not the right choice for you. A fully managed solution like Kinsta or WPEngine are good choices.
  • Google has a recommendation for various WordPress deployment scenarios here.
    • Depending on your load requirements (especially if they are large), you may need to consider a deployment on Kubernetes or App Engine. My blog isn’t that frequented, so I selected a Google Compute Engine based approach.
    • Google has a Marketplace / Click to Deploy template ready for you. It will step you through the process and deploy it pretty much in seconds.
  • Check your pricing: In Google’s cloud, for most basic needs the micro / free tier was a good fit for me. Note that the free tier only applies to certain regions, so make sure you select this when you set up your instance.

Step Two: Prepare your existing host provider for migration

  • Domain ownership: If your domains are owned by the current hosting provider and you want to transfer them out as well, you can perform a domain transfer to Google Domains. This is optional for just a WordPress deployment, but since I’m also deploying G-Suite for my domain, this was a good choice for me since the setup becomes one click too (for all G-Suite apps and email). This may take up to a week to execute, so keep this in mind.
  • Backup: There’s a fantastic WordPress plugin called the All-in-One WP Migration. It does pretty much what it says on the tin, downloading assets, code, themes, plugins, everything you need to migrate WordPress into a zip file that you can then restore on a new WordPress installation. Download this zip file and keep it safe.

Step Three: Setup the target server and start testing

  • Deploy the new WordPress instance using the Google Marketplace link. Select your compute instance type and region. For the free tier, make sure you select the right region and compute instance type.
    • Note down the usernames and passwords generated by the deployment script. Store it somewhere safe in case you need to rebuild your instance (Google stores it in metadata but you might lose if it you restore from snapshot).
      • Lesson Learnt: Take down your metadata / access details or recreate them on the new instance before you blow away the old instance…
    • Go directly to your WordPress instance and secure it with an account and password. Also, patch the operating system (with the usual ubuntu sudo apt-get update and sudo apt-get upgrade commands).
    • Cool tip: I deployed it to a larger instance type for testing and then downsized AND moved region after. You can do this by stopping the compute instance and taking a snapshot of the persistent disk. You can then create a new compute instance from the snapshot and change region and instance type accordingly.
  • Reserve Static IP: Assuming you are using a mono-instance deployment (that is no load balancer and multiple machine scenario), then you need to reserve a static IP. Note it down as you will need it soon.
  • Restore your WordPress deployment: Install the All-in-One WP Migration plugin on your new WordPress.
    • Your newly minted WordPress instance will likely stop working. This is normal.
    • Stop the server and take a backup by a snapshot.
  • Test your WordPress migration: On first load, you may be dismayed that your site isn’t working. This is normal, since the plugin migrates your WordPress instance as-is. There are a couple of steps you need to do in order to get it working again.
    • Change site URL temporarily to the reserved static IP:  Follow this WordPress codex guide and follow the edit wp-config.php section. I recommend this, since once you do the DNS update to the new IP address, you don’t need to alter the WordPress database. The php directives will override the settings in the wp_options table and you can uncomment them to do your service cutover.
      • Lesson learnt: There are a lot of pages on the internet telling you how to move URL, move IP address etc, but most of these are related to changing domain name. In this case, you will need update your WordPress database using SQL (notably on the wp_options table). Don’t just blindly execute Stack Exchange code into your WordPress Database! Read up the WordPress Codex first, I cannot stress this enough.
    • WordPress still not working? Other minor problems? Yes, there may be a few:
      • Re-establish .htaccess file: The migration plugin doesn’t move hidden files like your .htaccess file. Look at this guide to restablish it. If you update your permalinks setting, this will also initialise it.
      • Broken theme / WordPress general weirdness: Some themes are tied to older WordPress versions. On upgrading WordPress, you may find your entire site stops working. Now, you need to start debugging. Turn on logging and tail -f those logfiles and have a look around at what is happening.
      • Check your PHP memory limits: Depending on the complexity of your WordPress deployment, you may hit PHP memory limits. You can set this in the wp-config.php file as well.
      • Get wp-cli: OK, you might have ignored my advice and run random Stack Exchange scripts on my Database. To fix this, you can roll back your snapshot, or use wp-cli which has a fantastic search and replace function. Clone the git repo, install it, and give it a try.
    • Final WordPress touches:
      • Complete Google Analytics integration: Use a header plugin to insert the tracking javascript.
      • Update your theme and make the site pretty. I didn’t have enough time to replace my old incompatible theme, so I made some minor changes to a standard WordPress theme as an interim measure.
  • Setup billing and budgets in GCP: With the cloud you could end up spending a lot of money, so it is a good idea to set up some billing and budget limits. Don’t get caught out by large bills!

Step Four: Domain Transfer in complete. You may not get notified of this so keep an eye out or a calendar notice in your diary.

  • Setup Google Domains to point to your old service provider: This is essential, because if you don’t do this, your site will effectively be undiscoverable (including to Google Search). This is a sure fire way to destroy your referencing. Update A and CNAME records to your old service to keep yourself discoverable on the web.

Step Five: Cutover to new service

  • Final service testing: Check that the target server is functioning as required.
  • Final pre-cutover golden image backup: Stop the compute instance and take a snapshot.
  • Start the compute instance and retest by accessing the IP address in a browser.
  • Update the A and CNAME records in Google Domains to the new server IP address
  • Update the wp-config.php file with the domain name of your site, clear the cache of your browser, and your site should be functional.

Step Six: Shutdown old service

  • Shutdown your old service and turn off all auto-renewals. Confirm this with the support staff of your old provider and send them a note of thanks. They have been serving you well over the past few years after all!

Note that security of your instance is now your responsibility. If you are moving away from a managed service, this may require you to change your habits somewhat. Periodically update your WordPress instance to keep it secure. Periodically take snapshots and update the OS as well.

That’s pretty much it! There were a couple of gotchas that I found painful (i.e. I got wrong first time):

  • The Google free tier only applies to micro instances in certain regions.
  • If you do try a larger compute instance keep an eye on your budget.
  • If you are keeping your domain name the same, just override it in wp-config.php. Don’t touch the WordPress database!
  • Keep your DNS settings up to date during the entire migration.

Digital / Online Learning Content for 2017 – Early 2018

Complementing my recommended book list for 2017 – early 2018, there also exists a lot, a LOT, of online content for free that you can use to build your knowledge on modern IT topics. During a recent discussion with a Digital Architect in my company, when I asked him for advice on how to build knowledge his response was really telling: he told me that the best way to do it was to look online. He said that our company had tried to build internal knowledge capital but couldn’t keep up with the online community and market. So, we had stopped internal investment and directed our teams to look online. It was about at this time I started opening up my search online for good learning content.

The so called “Digital” word being bandied around so much and so broadly, it is difficult to target and or define what it is. For this learning track it is focused on the following:

  • Role: Modern Full-Stack Architect and developer i.e. someone that design a modern application AND build it AND test it AND build the infrastructure AND deploy it. That’s a lot of ANDs but in the Digital era, that’s what it takes. I do not shoehorn Digital into analytics, marketing, e-commerce, media or transactional platform applications — because I think these full-stack skills are required background to build innovative products in 2018.
  • Modern Technology as of 2018: i.e. scale out, internet-scale, highly redundant, highly available, transactional applications, analytics / ML / Big Data enabled, cloud based infrastructure.
  • Background: I’m assuming that you are working in an IT related role in a large organisation deliverying IT related projects.
  • Design Thinking Participant: How to get the keys to mindset of digital design. Often missing when you focus too much on technology, sometimes having a good idea counts for a lot.

The combination of the above two are extremely powerful, assuming that you continue to work in a decent workplace environment and learn through experience management, financial management, and leadership skills. The reason why they are so powerful are because they are particularly rare in IT. However, if you don’t want to open up and consider going full-stack or at least even consider learning a little about all aspects of software engineering, or have an “I’m an X and not Y” kind of attitude, then my advice here might be limited for you, but please have a look through this and consider it anyway as there may be links that may interest you.

Company Blogs: Most leading digital enterprises publish their points of view and information on how they build their applications and platforms. Why do they do this?  There’s no better way to prove your thought leadership than by sharing to the world how you went about doing this. I tend to prefer company blogs over provider sources (such as Google, Amazon, Microsoft) as they tend to give real world applications of technology.

  • Etsy’s platform blog: Covers their code, DevOps approach, and infrastructure.
  • Netflix’s Tech Blog: Really interesting blog focused on video delivery and retail infrastructure. Netflix create a lot of their own infrastructure so their approaches here are quite unique.
  • Spotify’s Tech Log: Spotify Tech blog includes a lot of information about their employees, their DevOps approach, and also how they have solved various infrastructure problems.
  • Instagram’s Engineering Blog: Instagram is a Python / Django shop and this blog talks about how they solve their python specific problems. They had a recent upgrade to python 3.5, which made waves in PyCon 2017, for many good reasons (see links below).

Cloud: Cloud providers have an incredible cadence of updates of their products. In order to keep up to date on what they can do and how they do it, you should check in every month or so into all three. Note that these blogs are focused more on existing customers of each cloud, so if you aren’t currently a customer or developer on their cloud, I’d recommend you go to their podcast or youtube channel and look at their keynote / yearly developer conference video instead.

Podcasts: A really good way to keep up to date on various sources, whilst in transport, cleaning the house, or when you can’t read.  I have only recently started doing this, so some of these recommendations may change. What I really like about the podcasts are that they interview real world people working on real world problems, unlike companies just trying to spin their platform (I’ve put in Google as an exception here because I find they have quite interesting content).

  • JavaScript Jabber: Interesting blog covering modern front-end JS topics. Given JavaScript is more or less the defacto front-end language now, they cover multiple frameworks and front-end considerations all in one blog.
  • Arrested DevOps Arrested DevOps: As it says on the tin, more general discussion on DevOps and DevOps deployments
  • Talk Python to Me Talk Python to Me: A great blog tackling some really great topics around python.
  • Google Cloud Platform Podcast: Some really interesting interviews, including Vint Cerf for example, talk more about more broad industry topics in ML / AI / DevOps, and gives you a little bit of an insight as to why Google is a little bit different to Microsoft and Amazon

YouTube Videos:

  • GOTO Conference YouTube Channel: Some amazing videos on code as a craft. Features videos on modern software architecture topics such as microservices, serverless, machine and deep learning.
  • Amazon AWS Youtube Channel Amazon Web Service’s channel gives you access to all their platform updates and customer success stories.
  • Google Cloud YouTube Channel: Google Cloud Platform’s YouTube channels have great videos from Next as well as demos of their unique platform tools like BigQuery.
  • I haven’t added Microsoft’s YouTube channel because I find their blog much more interesting and information dense! I recommend you go there instead.
  • Docker Con 2017’s playlist is full of little gems of knowledge on how to best use Docker.

Best of / Must See Individual Pod Casts / Videos: These are a collection of the best YouTube videos and pod casts that I have heard recently.

  • Instagram’s Gradual Typing Story: Instagram’s general typing of production applications by Lucasz Langa (who will take over version 3.8 as release lead). This is a must listen to for anyone in a code / system migration approach to hear how Instagram did this.
  • Instagram’s Pycon 2017 keynote: Instagram’s Pycon 2017 talk on how they did their migration to python 3. It’s funny, humble, and impressive.
  • IDEO’s Nightline Interview: It is a little old but gives you an insight on how a high-performance innovation design team works.
  • David Kelley’s TED Talk, associated with his book Creative Confidence. If you think you can’t do something creative, and/or have a fear of judgement that is holding you back, then watch this.
  • Spencer Kimbal’s overview on Cockroach DB  (one of my favourite recent tech startups). This is a little old, but it’s their first meetup video in SF, and is inspirational because their CEO explains their history and how they got into building this distributed and consistent database. Even if you haven’t heard of Cockroach DB after this video and have global / internet scale aspirations for your project, it will change the way you think about your database.
  • Amazon Web Services reinvent 2017 keynote: An overview of Amazon’s incredible cloud business and some customer testimonials showing the power of their cloud. It’s like the Superbowl meets IT (and it is a very enjoyable video to watch). Also, a great entry point to other Amazon videos, so you can explore other topics of interest.
  • Google Next 2017, technical keynote: Urs Holzle leads out a cast of other Googlers that present some of their most innovative products, like Cloud Spanner, which is a bit like a time machine into the future – only they have been using it already since 2012! For a computer scientist, this is super exciting stuff. Compared to Amazon, somewhat less Superbowl, and a little more corporate, but enjoyable nonetheless.
  • Amin Vahdat’s 2015 talk on Jupiter: Google’s Software Defined Network (SDN) deployment that delivers about 1.3Pb/sec of bandwidth… on merchant silicon (!!! Wow !!!). This video closes off a decade of SDN R&D and is really impressive. It really impacted me to see Google realise their vision of SDN – it underpins all of their cloud and their internet scale applications. I cannot underline how important this is. After watching this video, you can find other related links on YouTube about their networking projects.
  • The Code Whisperer, J.B. Rainsberger explains why integrated tests aren’t useful. After watching this video, go back and watch/listen to that Instagram python 3 upgrade again … and you’ll now understand why.
  • Michael T. Nygard’s talk on stability patterns and anti-patterns: Michael T. Nygard’s talk on stability patterns and anti-patterns, inspired from his book Release it! He’s a really great presenter and I’m so glad he put his learnings into a book.
  • Kevin Goldsmith talks about microservices at Spotify: Includes some information about how Spotify overcome their challenges given their high deployment cadence and some insight into their DevOps approach.
  • Josh Evans talks about Netflix’s microservice architecture: Josh Evans talks about Netflix’s microservice architecture. He also gives a little history lesson on how and why Netflix arrived at the microservice solution. He gives this talk after just stepping down from Netflix to take a career break.
  • Mary Poppendieck gives her quick summary of the future of software engineering: Mary Poppendieck gives her quick summary of the future of software engineering and summarises how we have moved from scale up to scale out, why federated architectures and software defined infrastructure and the cloud is the future. If you want to tell someone with hardly any IT expertise how and why the cloud matters, this is the video. Also if you are a developer, architect, or in IT in general, this tells you that on-premise and monolithic software is well on its way out – and so are you, if you work on that kind of thing.
  • Trisha Gee explains what it takes to stay ahead of the curve: Trisha Gee explains what it takes to stay ahead of the curve. She takes half the talk to explain new features in Java 8, but the really interesting part comes after as she explains what you can do to dramatically increase your learning of new technology or new practices.  Applies especially to those leading and looking for ideas for their teams or those in a team looking for ideas to grow outside the box / organisation. She’s another great presenter and has many other videos that are really thought provoking and entertaining.

Final stocking fillers: I give more text here on these because they might be a little less accessible to all, but I really do recommend them.

  • Louis Rossmann’s Repair channel Louis Rossmann’s Repair channel. Louis Rossmann’s repair videos are a lot of fun because he typically does it in real time. Not just any kind of repair but repairing highly integrated mainboards from laptops and smartphones. He fixes what the Genius Bar cannot fix. He is such a logical problem solver and just tells it like it is. He shares all his tips and solutions on the web and has 330k followers. In this example, he’s repairing a MacBook in real time and goes through the steps to identify the problem and fix it (not his best video as he’s fighting his new setup but it is recent). I find these videos strangely addictive and if you watch enough, you’ll get some pretty solid electronics repair tips as well. Sometimes he talks about his background and his life and they are as heart felt and down to earth as you can get.  A really decent guy.
  • Arduino’s Blog: Arduino’s blog is a gateway into all the wonderful things you can do with this open source electronics platform. I love electronics and I did start out an Electronics Engineering degree at university (but didn’t finish it), for my love of all things electronics. But I realised I loved building useful stuff with electronics and solving problems with it was more interesting than learning EE theory. Arduino gives you a gateway into the physical world allowing you to measure it and interact with it. Useful as a first step into what might be possible with the Internet of Things, a simple Arduino and sensor kit should be standard issue for anyone interested in computer science. If you have watched enough of Louis’ videos, you’ll know how to put those skills to practice building out your projects. And no, I don’t have a Raspberry Pi …. Yet.
  • Learning how to draw people: I recently for 2018 decided to learn how to draw since I think that visual communication is essential to design and IT architecture. You can find many great videos on YouTube and websites on how to draw, I will update this page once I’ve progressed on this front somewhat.

 

IT / Innovation Book Recommendations 2017-Early 2018

Here are my book recommendations for 2017, based on my learning objective to refresh my IT knowledge for the cloud / AI / ML / open source era. The catalyst for this learning effort was a real coincidence of multiple events happening all at the same time: my employer gave me training with MIT Sloan, I went to a conference on Artificial Intelligence, my employer started a marketing campaign about innovation and the new, and finally a friend joined a start-up and suggested some reading. All of these things crystallised in my mind and my current employer’s training left me somewhat unsatisfied and yearning for more, so I decided that I needed to build my own training program to give my IT skills a refresh.

A colleague asked me what I was reading so I thought I’d put it all together for others to consider.

Total Budget Required: About 500 EUR (plus I recommend buying a Kobo Aura One for another 220 EUR, bringing your total to about 720 EUR). In all cases, I am reading these books in their source language (i.e. English).

Books annotated with two stars ** indicate that I am currently reading them (i.e. not quite finalised my recommendation of them as of 01/03/2018).

Startup Spirit: What does it take to succeed as a start-up / challenger. These are highly referenced books and are very motivational / conversational in style – more for inspiration than material.

  • The Lean Startup, Eric Ries (18 EUR Amazon: Link)
  • Exponential Organizations: Why new organisations are ten times better, faster, and cheaper than yours (and what to do about it), Salim Ismail (8 EUR Amazon: Link)
  • The Innovator’s Dilemma, Clayton M. Christensen (18 EUR Amazon: Link)

Mindset: What kind of mindset do you need to succeed in innovation these are really great books to give you alternatives on how to think differently.

  • Thinking, Fast and Slow, Daniel Kahneman (9 EUR, Link)
  • Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale, Jennifer Davis and Katherine Daniels (19 EUR Amazon: Link)
  • X-Teams: How to build teams that Lead, Innovate, and Succeed, Deborah Ancona, Henrik Bresman (21 EUR Amazon: Link)

Leadership and Innovation / Design Thinking: Focusing on the IDEO founder’s trilogy of books, these give a great insight into how this design firm motivates a certain mindset, structures, and selects their team to build innovative designs for their clients.

  • The Art of Innovation: Lessons in Creativity from IDEO, Tom Kelley. (6 EUR Amazon: Link)
  • The Ten Faces of Innovation: Strategies for Heightening Creativity, Tom Kelley (10 EUR Amazon: Link)
  • **Creative Confidence, Tom Kelley (9 EUR Amazon: Link)

Product Design: What it takes to transform an idea and build it into a product. The Don Norman books have a lot of great examples on design hits and misses and are highly entertaining. On the other side, the Eppinger and Ulrich book pull this all together on present a very academic approach.

  • The Design of Future Things, Don Norman (8 EUR Amazon Link)
  • The Design of Everyday Things: Revised and Expanded Edition, Don Norman (8 EUR Amazon: Link)
  • Emotional Design: Why we Love (or Hate) Everyday Things, Don Norman (8 EUR Amazon: Link)
  • Product Design and Development, Steven Eppinger, Karl Ulrich (110 EIR Amazon: Link)

Executing Design in IT: What does it take to execute in a highly innovative and energetic environment.

  • Sprint, How to solve big problems and test new ideas, Jake Knapp, John Zeratsky, Braden Kowitz (13 EUR Amazon: Link)

Culture: AI and and computing history, to provide a little bit of history on how we got here and where we are going.

  • **The Master Algorithm: How the Quest for Ultimate Learning Machine Will Remake Our World, Pedro Domingos (7 EUR Amazon: Link)
  • The Soul of a New Machine, Trace Kidder (7 EUR Amazon: Link)

The Coal Face: To learn the new school, you need to know the old school. Everything you need to know about managing IT infrastructure and applications in the old school and the new.

  • **The Practice of System and Network Administration, Volume 1, Third Edition, Thomas A. Limoncelli (20 EUR Amazon: Link)
  • **The Practice of Cloud System Administration: Designing and Operating Large Distributed Systems, Thomas A. Limoncelli (22 EUR Amazon: Link)
  • Release It!: Design and Deploy Production-Ready Software, Michael T. Nygard (22 EUR Amazon: Link)

Innovation in IT: Now you know how to do it, and you have the idea, maybe you might want to build it. The spring book is now getting a little old, but still relevant for a lot of people. My current development track is focused on open source Angular, python, ruby, amoungst others in a containerised deployment, so take my recommendations here with a pinch of salt if you are learning other technical things at this time.

  • **Implementing Domain-Driven Design, Vaughn Vernon (20 EUR Amazon: Link)
  • **Learning Behavior Driven Development with Javascript, Enrique Amadeo (20 EUR Packt: Link)
  • js Design Patterns, Mario Casciaro (30 EUR Packt: Link)
  • Responsive Web Design with HTML5 and CSS3 Essentials, Alex Libby (17 EUR Packt: Link
  • NG-Book, Ari Lerner (39 USD: Link)
  • **Deep Learning: A Practitioner’s Approach, Josh Patterson (20 EUR Amazon:Link)
  • Spring in Action, 4th Edition, Craig Walls (32 EUR Amazon: Link)

Free Reading: For digital solutions, the main documentation is all online. Read these to keep getting the inspiration to push you to the next level.

  • Technical Blog posts:
  • Cloud Architecture and Solutions: Available from Amazon, Microsoft, and Google.
  • **Chaos Engineering, Building Confidence in System Behavior through Experiments, Rosanthal, Hochstein, Blohwiak, Jones Basiri. A free book, search for it on google.
  • **The Future Computed, Artificial Intelligence and its role in society, Microsoft. A free ebook, search for it on google.

I will follow up this post with my other free learning sources on blog, podcast, and other channels soon.

Creating a Pollution Sensor based on Plantower Particulate Sensors (PMS3003)

If you already have Arduino board hanging around and would like to create your own PM2.5 / PM10 particulate sensor, here’s a recipe on how to do this. Note that I have pulled together a lot of other resources from the web that were badly organised here into one document. My code is based heavily on code from DFRobot.com’s sample code (all references included at the end of this post).

Here’s a photo of the setup, with the 16×2 LCD at left, the prototype shield and mini-breadboard on the Arduino Uno, and the Plantower PMS3003 on the right.

What you will need:

  • Arduino Uno board
  • Plantower PMS 3003 sensor (or any other from their range, however note there may be code changes required – see later)
  • A 16×2 LCD display, compatible with the LiquidCrystal display Arduino library.
  • Highly recommended: prototype shield and mini breadboard, some jumper wires.

Given that these components are now heavily commoditised, I would recommend finding a reliable vendor on eBay for these components directly from China (i.e. one with a lot of positive reviews).

Setup instructions:

  • Obtain the sketch from my GitHub repository (see below)
  • Wire it up:
    • Wiring the Plantower sensor: If you didn’t buy the breakout board, I recommend cutting and stripping the wires and solder ends to them to use them with the breadboard. Check the PDF in my GitHub repository for the pin outs on the PMS 3003 sensor, unfortunately red and black do not necessarily correspond with VCC and GND.
    • Wiring the LCD: use the Sparkfun tutorial for guidance on the wiring plus also the resistor and pot required to adjust the LCD contrast.
  • Make the following customisations to my code to match your requirements:
    • Serial buffer read length constant (Line 12): This is crucial. Depending on your sensor, Plantower sends out data in either 32 bytes or 24 bytes. For example, if you have a PMS3003, you will need to set this to 23 (24 byte readout, remaining byte is for the header). If you have a PMS1001, you’ll need to set this to 31. Check the datasheets of the Plantower sensor to verify what you require.
    • LiquidCrystal constructor (Line 19): Make sure that the pins match the pin outs of your LCD. I didn’t use the standard pinouts of the tutorial, so change it if required.
    • Uncomment lines 62-82 to debug: These lines write values to the serial port of the Arduino, that you can monitor through the IDE. This is useful to debug the output of the sensor (and to verify that you are decoding its output correctly).
  • Important note whilst uploading to the Arduino: Since the Plantower sensor is wired into the TX/RX pins of the Arduino, this can interfere with uploading your sketch to the board. Unplug the sensor whilst you are uploading, then plug it back in after.

Common problems:

  • If your readout is zero, this probably indicates a problem with the serial buffer read length. I would download the datasheet and check Line 12. Uncomment lines 62-82 and use the Arduino Serial Monitor to debug.

My GitHub project with the code and data sheets can be found here:

https://github.com/pakmingw/ArduinoPlantowerSensor

References:

Fixing Cisco Linksys E4200 v2 Stability Issues

I have an old Cisco Linksys E4200 v2 and have decided to share my fix for various stability issues that are present in the hardware design. It’s still quite a useful little router for my purposes (even though now there are newer faster designs).

There are two problems that I have encountered:

  • Power Supply: The stock power supply is not the best and mine stopped working (the device would not power on after a year or two). The fix here is pretty easy, change the power supply.
  • Overheating / stability: There are a lot of reports of the E4200 overheating and becoming unstable. That is, after a couple of minutes or hours of functioning, it stops working.

My post deals mostly with the overheating issue.  A solution that I have found is to use a Raspberry PI heatsink kit to replace the heatpads and RF shield combo on the router. Other methods of freezing the router are not going to be long term solutions.

In order to install this, you need to open the router. There are screws on the bottom, hidden by plastic feet covers. There are then a set of plastic clips that hold the lid onto the router. Once open, the circuit board should look like the photo below.

There are two steps to the installation:

  • Install CPU Heatsinks: There are three RF shields. The two at the bottom are for wireless circuits and the larger one on the top is for the main CPU. Use a screwdriver to gently prise open the RF cover over the main CPU (it should click out). Take off the heat pads from the processors / surface mounted chips. Stick on the Raspberry PI chips as shown in my photo.
  • Create cooling vents: In order for the router to cool itself via convection, you need to add some cooling vents in the top cover. I recommend drilling holes into it in a grid pattern. You should cover not only the CPU area but also the wireless circuits as they are large generators of heat. Sure, it ain’t pretty, but it works.

Put the router back together and reboot it – it should start working.  Mine has been on for the past few weeks with some large sustained activity uploading to my AWS Glacier cold storage… and it works perfectly fine. Good luck!

MacBook 2017: First Impressions

I recently acquired a MacBook 2017. It has been over 7 years since my last new Mac purchase experience, so I thought I’d document the first impressions since my last purchase experience. (Full disclosure: I’m primarily a Microsoft Windows / PC user, with Office 2016 for personal and professional use).

In short: I’m really impressed. In under an hour, I had all my apps up and running, my files synched from the Microsoft cloud, and a fully productive, secure portable computer. The small software and hardware changes that Apple have made since 2009 are noticeably productive and pleasing. Finally, the computer lacks the platform lock-in feeling of other Apple products (which is a good thing).

 

In more detail:

  • Unboxing experience is really quite good (with some reservations): Next day delivery. The UPS box is designed to lift the MacBook box up and out as you open it. The laptop is the first thing you see, then the charger and instructions. The instructions are in a small envelope with an internal spacer in the envelope to make it a small box to fit the hole in the larger box perfectly: nice touch. The downsides then. The MacBook box is too large: they could have shipped a smaller adapter to make the box smaller. The MacBook box is only in English with a French sticker over it — it is a good hack, but not perfect.
  • Physical hardware: Small form factor and wow factor first up, until you pick it up and realise it is quite heavy. At first glance, not much has changed here since 2009, but then some subtle changes are really apparent.
    • Keyboard / Touch Pad The new keyboard feels unlike anything I’ve used before. It has a very positive click to every press but very little throw. The touchpad is classic apple, really reactive, smooth, precise. Good false thumb rejection. The force feature, a bit annoying as there is no allegory anywhere else. Instead of delete there is a power button … which does NOTHING as I type this. Total fail. I clock 107 WPM in 10 fast fingers at 97.8% in my first hour on the keyboard, so I guess this is a win for the Apple.
    • Screen: The screen is just wonderful. Great gamut, very sharp, and accurate colour from the get go.
    • Single USB-C Port: I’m a huge fan of this feature. One port to charge and to connect them all. It works with all my USB-C chargers thus far (15W and the supplied 29W)
    • Case polishing defect: The case is milled from aluminium and in my sample, you can see the milling lines on the wrist pad. You can only see it (and not feel it), but still. Apple needs to work a little on the finishing / sanding of their cases. A shame.
    • Laptop screen balance: The laptop is screen heavy, so it wants to lean backwards. Your hands on the wrist pads and the screen itself balances this out, but I imagine Apple agonised over this because they could have made the laptop heavier and more balanced (with more battery life).
    • Sound: The speakers are well designed and throw out quite a bit of noise for the size.
    • No fan / silent running: With the low thermal profile of these new Intel chips, the laptop is completely silent. Which is great apart from the click noise from the keys and mouse.
  • Software: Lots of small changes to OS X since Lion or whatever it was back then.
    • Setup: The initial setup wizard is almost the same, apart from the inclusion of AppleID / iCloud and FileVault.
    • Applications included: Pages, Numbers, and Keynote are now included in the base build. Maps is also there.
    • iOS features have managed to make it to OS X: Autocomplete, Siri, notifications, and others… These may seem like small features, but they are significant and show that Apple are sharing features cross platform.
    • Setting it all up: Unfortunately, common applications outside the Apple walled garden need to be side loaded from the web. Why not the App Store? Chrome, Firefox, Spotify, Skype, Office … I’m talking about you (this might not be Apple’s fault). As an added feature, post installation, OS X will ask you if you want to move the install to the Trash.
    • Safari: This is worth a bullet point on its own. It feels like it is more elegantly integrated into OS X than Firefox or Chrome. For example, as you enter in your credentials, you can save them but also grant access to other OS X features at the same time. I’m not entirely supportive of this but I can understand the convenience.
    • Retina Screen Rendering: I’ve got mixed feelings about this. By default, the magnification in the applications is huge, reducing the information density. But it also makes sense on the smaller screen. For the most part, Apple have updated all apps to be Retina ready, but there are some exceptions: like the flags for the keyboard formats and web applications. A small detail, but still.
    • Default screen font: San Francisco is a wonderful change from Myriad. It is more legible and cleaner (and arguably more so than Helvetica). It’s a nice touch that it is on the physical keyboard as well (although I do miss Univers).

Migrating from Amazon Kindle to Kobo E-Reader (Using Free Tools)

This blog post explains the migration from Amazon Kindle to Kobo E-Reader. I had purchased the Kobo independently of wanting to do such a migration (and use two devices) but I found tools online that facilitate such a migration (and also remove Amazon’s DRM in the process).

I believe such side loading is legal since I have purchased the book already from Amazon. I guess this is my only caveat.

My tools below are based on the Microsoft Windows platform; however you can use the Apple Macintosh versions of these tools. All of these tools are freely available.

What you will need:

  • Calibre e-Book Manager: A free e-book manager for Windows and Macintosh (https://calibre-ebook.com/). This tool will ingest the Amazon E-Books and convert them to the target profile desired.
  • DeDRM Calibre Plugin: A free plugin for Calibre that decrypts Amazon E-Books and removes DRM. This is an open source project, with pre-built plugins (https://github.com/apprenticeharper/DeDRM_tools/releases).
  • Amazon Kindle Reader for PC Version 1.17 or earlier: The version is important, as any version after 1.17 will output a file that is currently not able to be decrypted by DeDRM. Search the internet for an archived version of 1.17 (here for example).

Instructions

  • Download all the tools noted above.
  • Install Calibre e-Book Manager.
  • Install DeDRM Calibre Plugin, using the instructions provided. Within Calibre, go to Preferences, add a custom downloaded plugin by using the feature in Calibre and indicating the downloaded DeDRM Zip file release on your hard drive. Restart Calibre.
  • Run Amazon Kindle Reader for PC v1.17. Connect it to your account and download your books.
  • In Calibre, click Add Books and select all the books downloaded by the Amazon Kindle Reader for PC. These should be in Documents \ My Kindle Content, and should have the extension of .azw.
  • If DeDRM has been installed correctly, these files should be imported into Calibre. The titles of the E-Books will be in the list of books instead of the filename on your hard drive. Double click one to check. If you can open the file, the DRM has been removed.

At this point, you have successfully decrypted the Amazon Kindle E-Books and removed the DRM. If you have a Kobo, you can then convert and sideload the E-Book to your reader by doing the following:

  • Install the KoboUtilities plugin in Calibre by clicking Get New Plugin, finding it in the list and installing it. Restart Calibre.
  • Plug in your Kobo E-Reader via USB to the computer running Calibre. Click on KoboUtilities on the toolbar. It should be detected, so add it as a managed device.
  • Select the book(s) you want to transfer to the Kobo, and click Send to Device.
  • Calibre should perform the conversion and upload the book(s) over USB to the Kobo.

The Amazon Kindle E-Book(s) should now be on your Kobo. Congratulations!

Pentax K-30 / K-50 Black Picture Problem and Fix (Aperture Control Block)

Pentax K-30 and K-50 have a problem with the aperture control block mechanism within the camera that causes them to not actuate the aperture, hence giving only black photos. This is due to a small solenoid within the camera, that over time becomes more magnetized and fails to actuate.

The fix is simple: take out the moving horseshoe part from the solenoid and then file or grind it down to reduce the magnetic force imposed on it at rest. The filing works because it reduces the effective contact area at rest, hence reducing the amount of magnetic flux lines going through it.

This post is an addendum to the very useful video posted here by MBS TV. This will show you how to open the camera and also a good technique on saving the screws.
https://www.youtube.com/watch?v=1ZZoTUQvwBI

What the video doesn’t show you is how to do it for a K-30 nor how to extract the solenoid. To extract the part on the K-30, there are two tricks missing in the video:
• Unscrewing the solenoid: You should use the bit of a multibit screwdriver to access this. Normally there should be enough clearance and once the glue is removed, you can remove it by hand.
• Taking the solenoid out: You need to bend the top cover case slightly and simultaneously use a pair of tweezers to extract it.

Once it is extracted, you can grind it down. I have shown an example of the ground down part here. Remember to reassemble and test the camera before putting the case back on.

Photo 1: Unscrewing the solenoid. Note the screwdriver bit which comes from a mobile phone screwdriver set. If you search Amazon for this, you should find one. Also, you’ll probably find it handy for opening the camera in the first place.

Photo 2: Extracting the solenoid means you have the carefully bend the top cover by applying force as indicated by the red arrow.

Photo 3: Photo of the actuator in the solenoid once round down. you can see that grinding down the tips are the most important thing to do in order to reduce the magnetic flux going through the horseshoe.

 

 

 

Fix for Epson R3000 Ink Smearing / Dropping on Prints (Also works on R3880, R2880, and others)

A common problem with the R3000 (and other Epson printers that use the same print head), is that after a couple of years of service ink drops begin to fall on your prints. This is not due to a head strike, but actually due to with the PK/MK switch within the ink supply unit that gets blocked over time. After a while, especially if you use Photo Black (PK) only, such as myself, the MK line will block at the switch and then start leaking black ink.  This process will work for any other Epson printer with a PK and MK ink switch, i.e. 3880, 2880, etc.

UPDATE: Another user has confirmed that this works for the R3880. Great!

One solution is to switch to MK and then cut PK cartridges to fit into the MK hole – but this is hardly satisfactory. Another solution is to replace the entire ink supply unit, but this costs about 200 EUR and is labour intensive.  A final solution that I have tried (and works!) is to perform a complete line clean on the R3000, using refillable cartridges. The total cost is going to be for a set of refillable cartridges (~25 EUR), some flushing liquid (~25 EUR) and a set of cartridges (2 x PK + MK, i.e. about 100 EUR). Note that the cost of the cartridges you will require anyway, if you do perform an ink supply change.

Here is how it works:

  • Ensure you have a couple of days where you don’t need to print and a set of PK and MK cartridges in the printer at least 50% full, and as a minimum one new set at hand.
    • The 50% full ink cartridges will allow you to perform the PK / MK switch if you have a problem with your refillable cartridges.
    • You will need at least one new set of cartridges to recharge the lines and the supply unit. I would recommend two as a minimum.
  • Buy a refillable R3000 cartridge set and solvent cleaning set. I recommend the following (or similar):
  • Load the PK and MK cartridges with cleaning fluid. Load them into your printer. Store the original PK and MK cartridges in plastic cling wrap, making sure that you have covered the ink outlet holes and the whole cartridge.
  • Perform multiple ink changes between MK and PK. This will charge your PK and MK lines with the cleaning fluid. I would do this for a whole refillable cartridge capacity. Reload them and perform the switch again.
    • Perform intermittent print head checks to see if the lines are being flushed. You should see the full black head starting to get lighter and lighter until it becomes transparent (for PK and MK).
    • Open a word processor and print a page full of full black. You should see the print getting successively greyer and greyer. It should be somewhat grey at the end (due to the R3000 mixing other greys to print black), so remember to use the print head check to verify it is fully flushed.
  • Wait for 2 days. Switch to the other black ink. Wait for another 2 days. (i.e. leave the switch on each black colour at least 2 days on the cleaning fluid).
  • Perform another print with the word processor on full black. It should now be clean.
  • Unwrap the real PK and MK ink cartridges. Reload them into printer and perform a switch between PK and MK to charge the lines. Note that you will need about 20ml of ink to recharge the lines.

Epson sells a software utility for the R3000 to prime the lines to feed ink back through them. If you don’t have this tool, then you should use between 3-5 head cleaning cycles to do the same thing (one for PK and MK).

At this stage, the R3000 should now be fixed and no longer be dropping ink. Change to new PK and MK cartridges when ready.  Congratulations, you have just saved your printer! (Well, at least it worked for me …)

p.s. Do NOT use the syringe directly into the feed line — even though this may be recommended on some sites. This is for small format printers only, and if you do this, you may blow the lines into the supply unit permanently damaging the R3000.

Before Fix: Ink drops on prints due to problem with PK / MK switch. These are not head strikes.

 

After Fix: No more ink drops on prints!

 

Sanjay Gandhi Transport Nagar, Delhi, India

 

The Sanjay Gandhi Transport Nagar is a huge parking lot for trucks in the north west of Delhi. It’s the largest truck roadhouse in Asia, and handles between 2000-3000 trucks (and about 6000 truckers a day). With 6000 men moving in and out, there’s a lot of business to be done, so it’s like a miniature city with services for truckers and their trucks. And that means a lot of dodgy business, i.e. sex.

CSI’s role here is on safe sex education of the truckers and on providing a health service to them whilst they are passing through the transport nagar. To find out more about the truckers and the aid work here, we visited some of the organized events in the transport nagar by CSI.

Satish, a peer of CSI, was an owner of a transport company. On top of managing truckers, he ran 10 interpersonal communication (IPC) education sessions with the truckers that pass by every month. Of the 50-60 truckers, there are a couple who are not well, and he keeps a track of them. The IPC sessions involve group talks about sexual health and include a “body-mapping” exercise, which is an interactive drawing and discussion about the various body parts of a female and sources of STD. This exercise was designed to increase the engagement of the truckers as this kind of discussion would typically be taboo.

What is a typical trucker’s life like? I asked Kariji and Netabalu (two truckers who participated in Satish’s IPC session) for more information. They were currently delivering goods from Delhi to Visakhapatnam, a trip of about 1700km or 72 hours. Their sharing principle was simple: one guy drives, one guy sleeps. I looked inside Kariji’s new truck – there wasn’t a sleeping cab, they were just sleeping on the seats. They work away from home for many days at a time and do not see their families often. I asked Kariji (who had been a trucker since the 80s), what had changed. He told me that before, there was a real brotherhood between the truckers, there was less fighting, less theft. Things seemed like they were only getting harder.

CSI also ran 2 health clinics in the transport nagar and I visited Dr. Prem Sagar Gutta, to talk about the services he was being paid to offer to truckers. He would perform testing for STDs and interview them on their sexual activity and could get a lab report on their health in 24 hours (normally within the lay-in time for most truckers in the transport nagar). For the truckers, the consultation was free and medicine was provided at cost. This was an added incentive for them to go there and be checked out.

The next day, we returned to the transport nagar and saw a street play organized by some shop owners. The themes of the street play were around the same topics of sexual health and the lives of the truckers, to engage and educate them. At the end of the street play, another IPC body-map session was performed and a new set of truckers were educated and informed.

It was during the street play that attracted a lot of attention, that I noticed it also included the attention of some rag pickers and children on the street. On the way back to the office, I noticed the underside of all that we were educating: the reality of women and children on the street. Two kids were collecting garbage and carrying magnets on rope to collect scrap metal, a group of ladies were taking shelter in the 42C heat under a truck together, and a woman was leading 4 children through the roads of the transport nagar.

I spared a thought for the truckers, those in the transport nagar serving them, and those being exploited and trying to find a way to just live. A just and equitable solution to the problem was just one step too far away. Education and change was happening, but it was going to take time.

WWW: http://www.childsurvival-india.org/