Blog

Clone Live Windows Disk To USB

I have been using a Dell XPS 13.3 for the last few months and am loving it. It’s portable, light, got an amazing FHD screen with almost zero bezel and fast. Until recently it has been perfect, however I have started using virtual machines for work projects so the 256GB SSD quickly started to take strain. I thought I’d just get a larger SSD, make a clone and will be up and running in no time. I was wrong, very wrong!

I went for a Samsung 500GB M.2 SSD and because my XPS only has one SSD socket, I bought a M.2 SATA to USB 3.0 enclosure too. That was the easy part done. The ‘fun’ was only just beginning.

When looking at the original SSD in Windows Disk Management , you will notice that there are a few hidden partitions where Dell keeps OEM data, recovery partitions and the main C:/. I needed to clone all the partitions over to the new SSD which was connected via USB 3.0.

Clone Windows SSD to External USB - original ssd

I will start with the program that I ended up using, and then mention a few that I didn’t and why they didn’t work for me. This is not a step-by-step guide, as I am already working on my new SSD. It’s more for information and choosing a program to use.

Macrium Reflect

I came across Macrium Reflect by chance and I am so glad that I did. It was by far the easiest to get working.

Clone Windows SSD to External USB - Marcrium Reflect

All you need to do to get going, is install the software, I included Windows PE support for drivers in case they were needed. Connect your new SSD to a USB port. If the SSD is brand new and never been plugged in before, you will need to initialise the SSD using Windows Disk Management. Once the SSD is initialised and has a drive letter assigned to it, you are good to go.

Once Macrium Reflect is installed and your new SSD is plugged in, run the software which will look like this.

Clone Windows SSD to External USB - Marcrium Reflect main window

I made sure all other applications were closed and once ready, selected ‘Clone this disk…’.

The source disk will be the disk currently running and the destination disk will be the new disk connected via USB. Follow the prompts selecting the options that suit your installation and then run the clone. The clone I did on a 256GB SSD took around 1 hour to complete. After each partition is complete, a Windows Explorer window will open showing the newly cloned partition.

Once the clone is complete, shut down Windows, replace the SSD in your laptop with the newly clones SSD connected via USB and boot your laptop. My first boot took a few seconds longer than normal, but everything started perfectly.

Because my new SSD was double the size of the old SSD, there was an uninitialised partition at the end of the SSD. Using Windows Disk Management I initialised and assigned a drive letter to that partition and now I have loads of extra space to use.

Clone Windows SSD to External USB - Windows Disk Management

Programs that did not work for me

I am going to quickly cover the programs that did not work for me.

First I tried DriveImage XML as a many articles that I read suggested this application. The main reason I decided not to use this was you are only able to clone a single partition. I’m sure this works well when you have done the initial installation yourself into a single partition, but most PC’s/Laptops’s come with OEM installed Windows which includes multiple partitions.

I then tried EaseUS Partition Master as it was also a free application. This required making a Linux boot disk which you boot off and then you select which is your source and destination disk. Unfortunately when I booted off the boot disk, the SSD connected via USB was not recognised.

Thirdly I tried Acronis True Image 2015 which I bought a licence for as disk cloning is not supported in the trial version. Acronis True Image also uses a Linux boot disk to clone from one disk to another. When I first tried it, I had the same problem as EaseUS Partition Master where my the SSD connected via USB was not recognised. After installing the multiple media packs and spending hours on forums, I decided to look for another application as the workarounds mentioned seemed totally unreasonable for a paid, licensed application.

Conclusion and Solution

After many hours and multiple reboots, I finally found Macrium Reflect which worked first time, perfectly. From download to rebooting on my new SSD took less than 90 minutes, and I didn’t have to worry about making Linux USB boot disks. I highly recommend this program for performing clones on Windows disks. I am going to look at using it for my file and systems backups too. They also offer a free version that is loaded with features for non-commercial use.

As usual if you have any questions or comments, feel free to get in touch via Google+ or email.

Greg

myRIO: FPGA SPI Communications

I have had a National Instruments myRIO for a few months but have not had a chance to use it yet. I bought it so that I can get some experience working in LabVIEW Real Time and with the LabVIEW FPGA.

I started off using the default FPGA build and express vi’s but quickly ran into a problem when I tried to use the SPI express vi. Each time I tried to configure the express vi, LabVIEW crashed. After a while this got extremely irritating so I started looking at the myRIO Custom FPGA Project.

myRIO Custom FPGA Project

To use digital IO this project worked really well. Again the problem came when I wanted to interface to an SPI device. I have used SPI a lot in previous projects so understand the concept and how it works. Despite being a project template, with a documentation folder, nowhere does it explain how to use and implement the custom FPGA build. If there is further documentation, please let me know so I can go through it to get a better understanding of the project.

One more small rant before I get on to a solution you can use, is the SPI naming used in the myRIO Custom FPGA Project does not conform to the norm. Again, if I have read and understood it incorrectly, please let  me know.

myRIO SPI Naming Convention

While looking for some documentation on the myRIO project, I came across this example which explains a bit about how to write an SPI driver and also has an example. This example is written for the sbRIO but can be very easily modified to run on the myRIO and also has a nice, simple API. The last section of the NI article outlines in a few lines how to recompile for a different FPGA target. Here is a more detailed description with a few extras that I have added. Firstly, download the example project here.

myRIO SPI Sample project

These are the steps that I followed to to get it working on a myRIO. I’m sure the steps will be very similar when moving to any other FPGA target.

Add your target to the project.

myRIO SPI Add new FPGA target

myRIO SPI Add new FPGA target 2

Once you have the myRIO target added to the project, copy the FIFO‘s and Example_Host SPI Dual Port.vi from the sbRIO FPGA Target to the myRIO FPGA Target. Make sure to copy them to the FPGA Target and not the RT Target. Then copy Example_Host SPI Dual Port.vi and FPGA Reference.ctl from the sbRIO RT Target to the myRIO RT Target.

For my project I decided to use Connector B. Rename the IO that you are going to use for the SPI communications. Your project should now look like this.

myRIO SPI new FPGA Target

Next we have to edit the FPGA code to work with the new references set on the myRIO FPGA. Open Example_Host SPI Dual Port.vi. I have deleted one of the SPI ports as I only need one.

Open FPGA SPI_SPI Port.vi and change the references in the FPGA IO cluster to point to the IO we configured in the previous step. Right click on each reference and select Conifigure IO Type…

myRIO SPI configure new IO references

Select the corresponding reference and then Replace All.

myRIO SPI configure new IO references 2

Do this for each reference. You will notice that when you change the CS reference, the vi breaks. We will fix this next. The reason the vi breaks is that the CS line on the original sbRIO FPGA code was set to a line and not a port. When changing form a line to a port, the element is now boolean and not U8.

The two changes that need to be made to FPGA SPI_SPI Port.vi are removing the Index Array in the Reset CS case and add a Number to Boolean Array and Index Array to the Set CS case. This will test the state of the LSB which will be the CS state.

myRIO SPI configure FPGA CS

myRIO SPI configure FPGA CS 2

Save and close the vi. In Example_Host SPI Dual Port.vi delete the FPGA IO cluster and then add it again by adding a constant to the subvi. This will update the references that have been changed. Save and close all open vi’s. Remove the original sbRIO Target and save the project.

myRIO SPI Removing sbRIO Target

You are now ready to compile your myRIO SPI project. Right click on FPGA SPI_FPGA Top Level.vi and select Create Build Specification. When compiling there are a few choices for where you want to compilation to take place. To compile this locally it took me around 50 min, compared to compiling on the NI Cloud Portal which took less than 10 min.

myRIO SPI NI Cloud Compiler

To access the NI Cloud Portal, you need to register an account as your normal community account does not work here. Once registered you will need to verify your account through an email confirmation. You also need to enter a Product Activation Code. This is not the code that you use as your LabVIEW license. I had to call NI and get them to generate a 20 character code that I entered. You will need an active SSP to get a code generated. The time it takes to call NI and get the code is less than one compilation so is totally worth it.

Once the FPGA compilation is complete, open Example_Host SPI Dual Port.vi. You will notice that the run arrow is broken. To fix this we need to link Open FPGA VI Reference to the correct bitfile. Right click on Open FPGA VI Reference and select Configure Open FPGA VI Reference…

myRIO SPI Configure FPGA VI Reference

Select the Build Specification and then select Overwrite in the dialog that pops up. The vi should now be executable.

Run the vi, select your slave device SPI settings and mode and then press the Write button. You will notice data will be read back.I have a LM74 temperature sensor connected which uses 16 bits SPI mode 0.

myRIO SPI RT Running

There you have it. SPI running on a myRIO FPGA with an API which can be used on an RT host.

As usual, if you have an questions or advice, please feel free to get in touch.

Download myRIO SPI Example Code

Greg

SSH with LabVIEW

Please note that this project is no longer being supported and does not support some of the newer connection methods.

I was asked last week if there was a way to interactively communicate with a remote Linux client. The only way that I knew of was to create a temporary file with the commands and then use Plink (a part of PuTTY) to execute the file of commands on the remote machine.

LabVIEW SSH plink PuTTY

This works well when you want to execute a list of commands and are not interested in the response from each command. If you want the response from each command, then you will need to call this method for each command. This is not very efficient as each time this method runs, it will log in to a different session before executing the command.

This method will also not work if you need to run custom applications on the remote client where the session needs to stay in memory.

I did a bit of searching and didn’t come across any LabVIEW solutions that suited my needs. I did find a really good C# library that looked like it would work. It is still actively supported and the latest version is only a few months old. (All good so far)

As the library is massive, my plan of action was to use the library to write my own C# dll that would do what I wanted it to do. All that I needed my library to do was connect to a remote client, execute commands and return the response and then disconnect when finished.

C# class

I decided to put everything into a class so that I can add to it in the future. There are still a few features that I want to add but will leave them for another time.

connect()

You need to create the following objects that will be used by the various parts of the application.

  • SshClient – used to connect and disconnect from the client
  • StreamReader – this will be used to read what is returned from the client
  • StreamWriter – use this object to write to the remote client

public void connect()
{
sshClient = new SshClient(remote, user, password);
sshClient.Connect(); //connect to the client
stream = sshClient.CreateShellStream("dumb", 80, 24, 800, 600, 1024);
reader = new StreamReader(stream);
writer = new StreamWriter(stream);
writer.AutoFlush = true;
}

Once these references are created, they are stored in public properties.

executeCommand()

The execute command methods writes a string to the StreamWriter object.

public void executeCommand(string command)
{
writer.WriteLine(command);
writer.AutoFlush = true;
}

Once the string has been written, the buffer is flushed.

disconnect()

The disconnect() method closes the session and disconnects from the client.

public void disconnect()
{
sshClient.Disconnect();
}

The class only contains these three methods. There is one more step that needs to be taken and that is to read the StreamReader object for responses from the client. I have left this out of the class and leave it up to the application to do the reading.

readReader()

private void readReader()
{
try
{
int i;
for (i = 0; i < 3; i++)
{
Thread.Sleep(500);
if (ssh.stream.DataAvailable)
{
tbOutput.AppendText(ssh.reader.ReadToEnd());
tbOutput.ScrollToCaret();
Thread.Sleep(50);
}
tbDebug.AppendText(i.ToString());
}
}
catch(Exception ex)
{
MessageBox.Show(ex.Message.ToString());
}
}

In the readReader() method, I monitor the DataAvailable flag. I have had to play around with the number of iterations of the for loop as different tasks take different amounts of time. Some examples that I came across only call this method once, but I found that some data is then missed. Setting the iteration too high causes unnecessary delays and setting it too low does not allow all the data to be read. There are other methods that can be used, but I still need to give those a try.

Implementation in LabVIEW

UPDATE: I have uploaded a video of a LabVIEW application being used to to communicate with LinuxLite running in a virtual machine.

LabVIEW SSH front panel

My main aim was to get a LabVIEW application working so once I had my class written, I moved into LabVIEW mode and connected everything up.

Using the .NET palette, I was able to call the constructor, read/write properties and execute methods on the class.

LabVIEW .NET pallette

I wrote a simple state machine application to test the class. All the references are stored in a typedef cluster which is shifted around each state. On the front panel there are buttons that generate events to call each case.

Constructor and connect to client

LabVIEW SSH constructor
LabVIEW SSH connect

Execute commands on the remote client

LabVIEW SSH Execute Command

Disconnect from the client

LabVIEW SSH Disconnect

Read the StreamReader object

LabVIEW SSH readStream
LabVIEW SSH readStream 2

With this application, I am able to connect to a remote client, execute commands as I would in PuTTY and then disconnect when I am done.

There are a few things that I still need to sort out. The main one is that terminal characters are being returned. These are not seen in PuTTY as PuTTY removes them from the string that is shown. The other is that the prompt is being written twice. From what I have read this is caused by the client echoing what it receives. I still need to look at this too.

In future versions, I also want to add SFTP functionality. Being able to upload and download a file will be quite a nice feature. I also want to add different login options. Being able to login using private keys will also be useful.

As with all my posts, you can download the LabVIEW code. The download link contains a slightly modified library from the above post. If you have any feedback, please feel free to contact me on Google+ or send me an email. The LabVIEW application have been tested on the default Raspbian image running on a Raspberry Pi B.

Download LabVIEW LabSSH & LabSFTP Library

Greg

Executing Perl scripts in LabVIEW

The project that I am currently working on requires Perl scripts to be executed through a LabVIEW application which configures the UUT. As Perl is not natively supported by Windows and LabVIEW (I might be wrong here), other tools are needed in order to execute the scripts correctly.

As the scripts were developed on Linux, there was never an issue running them before developing the LabVIEW application.

Running Perl scripts on Windows

First we need to be able to execute the Perl scripts on Windows, then we can move on to LabVIEW.

The tool I am using is called Cygwin.

Cygwin is:

  • a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows.
  • a DLL (cygwin1.dll) which provides substantial POSIX API functionality.

To install Cygwin, these are the steps I took:

  • Download and install the your require for your PC
  • Select the Root Directory (C:\Users\gpayne)
  • Select the Local Package directory (C:\cygwin)
  • Select Available Download Sites: http://cygwin.mirror.constant.com
  • When selecting packages make sure you select perl(under Interpreters Group) and ssh (under Net Group) packages.
  • Ensure to add the shortcut to the desktop (We will use this later)

Once installed, running the shortcut on the desktop will open a terminal.

Cygwin Mintty terminal

The pwd command will give you the location and should be the same as set by the Root Directory above.

Create a Perl script in that directory. As an example I have created a Perl script that just prints a number for 5 seconds and then exits. To run the script, enter ./Test_Script.pl in the command line and watch as it runs.

#!/usr/bin/perl -s

my $number = 5;
while($number--)
{
print "$number\n";
sleep 1;
}
exit 0;

Cygwin Mintty terminal running Perl script

Running Perl scripts in LabVIEW

To execute an external application from LabVIEW, one way is to use the System Exec vi. This covers executing the application/script, however Windows is still not able to run a Perl script if it is just called.

The first thought that popped in my mind was to use a batch file.

Using Windows batch (*.bat) files

After doing a bit of research, I found a simple way of calling the Perl script, using bash.exe from a batch file.

@echo off

C:
chdir C:\Users\gpayne\bin

bash --login -c "./Test_Script.pl"
exit 0

This batch file, navigates to the bin directory that was installed by Cygwin and executes the Perl script using bash. The batch file then exits when the script completes.

To accomplish this in LabVIEW is really simple. Just pass the path of the batch file into the command input of System Exec and run the vi. The file runs and the output is reported.

LabVIEW system exec batch file Perl script

LabVIEW system exec batch file Perl script output

This is great because now we can run a Perl script on Windows through LabVIEW and get the Standard Output and Return Code once the script is complete.

This worked well until I started seeing times when the batch file would not exit. I tried all the escape codes that you can call on exit but nothing worked consistently.

The Perl script would execute perfectly every time when run in the terminal, but when run from the batch file and LabVIEW, it would not.

This was not acceptable so I had to look for another way to execute the script successfully every time.

Using Cygwin and Mintty.exe

As the Perl script worked perfectly when run in the terminal, I started looking at ways of replicating it from within LabVIEW. First was to look at what was run when you double click the desktop shortcut. The target points to C:\Users\gpayne\bin\mintty.exe -i /Cygwin-Terminal.ico - and this is what got me started.

I noticed that the application that is run is called Mintty.exe so I took a look at the help file. To get the help file, just type mintty --help at the terminal.

Cygwin Mintty terminal help

I was not interested in the --icon option, but the --exec option looked promising. This got me thinking of what I wanted to execute using Mintty.exe. I needed to execute my Perl script using bash.

So I started to build up my string that I would write to System Exec in LabVIEW. After some trial and error, I was successful in executing the Perl script. This is what I came up with.

C:\Users\gpayne\bin\mintty.exe -e /bin/bash -c '/home/gpayne/Test_Script.pl'

The bash help files are also helpful so from the terminal type bash --help or bash -c "help set".

This would execute the Perl script with bash running in Mintty. This was all good until I noticed that the standard output was not being reported back to LabVIEW. I needed the standard output as this is what I parse to determine if the script was successful or not.

This is very easily solved by piping the standard output from the script to a file and then get LabVIEW to read the file once the script exits. This does add an extra step, but by executing the script in this way, it runs and exits cleanly every time, being much more reliable than using the batch file. This is my final string that was used.

C:\Users\gpayne\bin\mintty.exe -e /bin/bash -c '/home/gpayne/Test_Script.pl > StandardOutput.txt'

 LabVIEW system Mintty bash Perl script output

After trying both methods, I settled on using Mintty instead of Windows batch files. They seemed a lot more reliable over many executions.

This method can surely be expanded to running Python scripts too, which I will leave for another day.

As usual, please direct any questions, comments or tips to my email or on Google+.

Greg

LabVIEW: Built Application Version Property

A LabVIEW project that I have previously worked on had many different components, each with there own version numbering. A global variable was used to READ the version numbers when the application was run.

This worked well, but became rather cumbersome when we had to manually change the global variable each time we modified or built the application. This became apparent and very frustrating when a build would be run, which could easily take an hour or two to complete, and only when running the application did we realise that we forgot to update the global or save its default value.

I thought there must be an alternative so went about exploring the application properties and methods. As it turned out, it is really quick and easy to pull the application version number from the built executable in LabVIEW.

LabVIEW Application Version main

LabVIEW Application Version subvi block diagram

Only two property nodes and one method are needed to get the version number from the built application. First you need to get the type of application that is being run. An enumerated type is returned and if this vi is run from within an application, the App.Kind property will be “Run TIme System”. In this case, pass in the path to the executable, and then get a reference to the FileVersionInfo. The FileVersion property can then be read. This will be the same value as set in the Version Information setting of the Application Properties.

Application Version Information properties

The App.Kind property can be used in a number of ways when you want specific settings to be applied only when running in certain environments.

I hope you find this useful and as usual the project used in the post can be downloaded below.

Download Build Version Property

Greg