A graphical interface that enables humans to interact with machines and vice versa is an HMI (Human Machine Interface). Typically, HMIs are used to configure, control and monitor the machines.
Here are some examples:
- HMIs for intelligent sensors and sensor networks
- Web-based HMIs for remote configuration and monitoring of applications
- Industrial HMIs for process plants
- HMIs for embedded devices such as routers, network gateways, smart devices, etc.
- HMIs on hand-held devices such as smartphones and tablets
Today, there are several challenges in building an HMI:
- Choice of technology – There are several HMI development tools and technologies to choose from, i.e., technologies for desktop applications, mobile apps and web-based apps.
- Flexibility and scalability – It’s crucial to consider the ability to add new functionalities demanded by customers, without affecting the existing functionalities.
- Modularity – Developers should follow a modular approach so that different teams can focus on building different components of the HMI. This also ensures maintainability once the products are rolled out into the market.
- Multi-platform – Due to a huge user base with different needs, it is important for some companies to offer solutions on multiple platforms such as Windows, Linux, Mac (in the desktop space), iOS and Android (in the mobile space) and Blackberry. Of late, mobile HMI applications are increasingly becoming key focus areas for certain manufacturers.
- Multilingual – Nowadays, the workforce is more globalized. It is very common to find people from multiple geographies working together, either from one location or multiple locations. For eg., the process engineer of a plant and the plant worker may be from different linguistic backgrounds. Hence, the ability to provide HMIs in multiple languages becomes very important.
Basic Components of HMI Software
The building blocks of HMI software are as follows:
- User Interface (UI) – The user interacts with the application through the UI, performs different operations and accesses various types of data.
- Display component – Used to display the graphical data that is captured from the embedded system.
- Standard tools – This includes different buttons, labels, fonts, etc.
- Customized tools – These will be needed if we are developing a UI that requires customized components.
- Embedded system – It captures the data and sends it to the UI layer.
- Back-end server – This is an optional component, and it is used for saving the data coming from the embedded system or for storing configuration parameters. It is not needed in two scenarios – when there is real-time data transfer or when the data is not required to be stored anywhere.
- Communication layer – This works as a channel between the UI layer and the embedded system. This layer implements different types of communication protocols like TCP/IP. The UI layer and the embedded system should understand these protocols to communicate with each other.
What are the Challenges in HMI Development?
Listed below are the main challenges:
- Selection of technology and platform in HMI development
- Increase efficiency and productivity of the system with added functionality
- Performance, scalability and integration with other apps
Key Factors that Affect HMI Design and Development
- Analysis of product specifications and features – to understand the operations that need to be performed
- System architectures, standards and platforms – to check for dependencies or constraints
- Ease of use
- Multilingual support
- Ease of implementation and administration, while ensuring scalability
- High performance 2D and 3D graphics
- Total cost
Categories of HMI Development Tools or Software
- Proprietary (Provided by the manufacturer and it only works on the proprietary hardware)
- Open Software (System built on open technologies)
Advantages – It is very easy to use as all the APIs are provided by the manufacturer; quick development phase.
Disadvantages – Software will only run on that specific hardware platform.
The HMI application can be deployed on different hardware platforms, i.e., the software is not bound to one type of hardware. The software environment may be open source or licensed. For e.g., Qt HMI that can be deployed in Linux, Windows and other operating systems.
Advantages – Openness in the design process, huge software community support.
Disadvantages – Advanced programming skills and fine tuning ability is required. For e.g., There are scenarios where different open source libraries need to be included for the communication layer. In such cases, the developer must evaluate different libraries and implement the relevant interfaces.
Leading HMI Development Software Frameworks/Environment
- Qt – This framework is developed by Open Source Project. The main programming language used here is C++. All the bindings for other languages are also available.
- .Net – This framework is developed by Microsoft. The programming language used is C#. All the bindings for other languages are also available.
- iOS – This is a mobile platform offered by Apple. The programming language used for iOS development is Objective C.
- Android – This is a mobile platform developed by Google. The programming language used for Android development is Java. But it also provides good support in native languages with the software development kit.
Client Server Based and Standalone HMIs
Here’s a simplistic view of the client-server based HMI applications and standalone HMI applications:
- Client Server Based HMI
- Standalone HMI
Consider that there is a data source like an embedded system, producing the data. There are clients which are accessing this data and the users are performing different operations using this client. In between, there is an HMI server that holds the responsibility to access data from the back-end system and perform data manipulation, convert data into readable formats, and send this data to the client HMIs.
The client HMIs can be web-based or desktop HMIs.
The other type of HMI framework is the standalone HMI. In this application, the HMI’s UI part is directly accessing the data source. It performs the required modification of data so that the data can be properly displayed on the UI.
Cross-platform HMI
Cross-platform HMIs are developed for multiple platforms. So, if the HMI application is developed on one platform, only a small amount of modification is needed to deploy it on another platform.
For example, HMI Application on platforms:
- Windows
- Linux
- Mobile
Cross-platform HMI Comparison
| Parameters | Qt | .Net | iOS | Android |
| Platforms | Windows, Linux, Mac, Mobile platforms | Windows | MAC, iOS | Android |
| Application Deployment | Redistribution of Qt environment along with the application is possible through:
1) Static Linking 2) Shared libraries |
.Net framework or .Net Redistributable package is needed to deploy the HMI application. | Uploaded on App Store.
After review and approval by Apple, it is deployed on App Store. It can then be downloaded from App Store. |
Deployed on Google Play.
Download from Google Play.
|
| Application Development Cycle | Fast development cycle on all platforms by using cross-platform Qt creator IDE | Fast development cycle due to sophisticated IDE Visual Studio | Fast development cycle due to sophisticated IDE Xcode | Multiple IDEs
Development time depends on knowledge of IDE |
| Memory Management | Memory management by programming level handling | Automatic garbage collection
Can also be handled at programming level |
Memory management is based on reference counting mechanism | Automatic garbage collection
Can also be handled at programming level |
| OpenGL (Open Graphics Library) Support | Very good support provided by the framework using QOpenGLWidget | Support by wrappers | Very good support by using OpenGL ES API | Very good support by using OpenGL ES API |
OpenGL (Open Graphics Library) Support: If we are developing a graphics intensive HMI application, then this is a very vital parameter because this is used for visualizing 2D and 3D data. It is also a multi-purpose open standard graphical library that supports application for 2D and 3D digital content.
For iOS and Android, OpenGL ES API is OpenGL for embedded system. It is a simplified version of OpenGL that eliminates the redundant functionality to provide a library that is easier to use and implement.
What is a Multilingual HMI?
- Multiple languages supported in a single HMI application. The languages can be easily changed.
- HMI for a global audience.
- Multiple users can access the same system in their preferred languages.
Challenges and Architectural Considerations for Developing Multilingual HMI
Major challenges:
- Limited memory to store font-related information (bitmap font, outline font, etc.). This is particularly relevant if the platform is limited in terms of memory. In that case, we should take this into consideration when developing the HMI application.
- CPU processing power
- Size of the script engine
- Small resolution of the display screen – How different components/units will be visible on different types of display screens should be carefully analysed and planned.
Architectural/design considerations when we opt for multi-lingual HMI application:
- Plan and design the internationalization requirements well in advance – This will help us understand the different languages the HMI will have to support
- Understand user interface limitations
- Script engine functionalities and font information
- Ensure that there is a proper display for all the supported languages
- Understand available memory and its usage
Multilingual HMI Comparison
| Parameters | Qt | .Net | iOS | Android |
| Internationalization | High level of support by the framework | High level of support by the framework | High level of support by the framework by using application’s preferences | High level of support by using application’s preferences |
| Change Application Language | 1) Change application language without change in Operating System language.
2) No application restart required. |
1) Change application language with/without change in Operating System language.
2) No application restart required. |
1) Change application language with/without change in Operating System language.
2) Application needs to be restarted. |
1) Change application language with/without change in Operating System language.
2) No application restart required. |
In some cases, the operating system language is different from the HMI application language. The info under “Change application language” parameter is useful to bear in mind in those cases.
How Multilingual Concept is Implemented in Some of These Frameworks
- Qt – In Qt, we use the tr() tag for all the text we want to change to a different language.
- iOS – The steps to create an iOS application supporting multiple languages are as below:
- For each supported language, create a list of key-value pairs. Keep the key unique for each text.
- Enable multilingual support in the project configuration.
- Change language – This can be done in two ways:
- App localization – Change language in the iPhone’s OS settings and start the app
- Multi language support – Give provision to change the language within the app and restart the app
- Every app has a preference file from which it reads the default values like language, time, etc. While changing the language, this preference file is updated. The app reads this preference at the time of application initialization.
HMI Development Tools – Plugins
A plugin is a powerful custom extension for the HMI application. They are pieces of code, often DLLs or JARs.
Consider that we have a main HMI application. If we want to integrate certain components within the application, without affecting the functionality of the main application, we can use plugins. This is like an add-on to the main application.
Plugins are loaded at runtime itself and have partial control over the execution of the host program.
Benefits of Plugins
- Extensibility – The application can be dynamically extended to include new features.
- Parallel development – Since features can be implemented as separate components, they can be developed in parallel by different teams. This helps in reducing the development cycle timelines.
- Clear development direction – Since the plugin framework provides a well-defined interface and documentation for plugin writers, developers have a clear roadmap for development.
- Simplicity – A plugin typically has one function, and so, developers have a single focus.
Comparison of Plugins Across Platforms
| Qt | .Net | iOS | Android |
| High level of support by the framework.
Q_DECLARE_INTERFACE macro to let Qt’s meta object system aware of the interface |
Support in framework by using MEF (Managed Extensibility Framework), a component of .Net 4.0 | Not supported | No standard support by the framework
Can be implemented by using interfaces and plugin architecture |
Conclusion
HMI applications are undergoing rapid technological changes. Newer functional areas and business models are being introduced due to wired or wireless communication channels.
The early product development phases (concept and design) will have a big impact on the overall development of the HMI application. The licensing cost of development and deployment of the HMI application plays a sigificant role in technology selection.
New HMI development tools and technologies increase flexibility in adding functionalities and provide a rich look and feel. Concepts like plugins enable faster development cycles and flexible applications.
Multilingual support can break language barriers, and the same product can be operated in multiple languages across various geographies effortlessly.
