Generate EXEs solving MILPs using PyInstaller in Python

When developing software prototypes to optimize linear problems, it is sometimes desirable to provide the customer with an app to avoid the need for them to install a full python environment with all dependencies.

In python, there is a tool called PyInstaller for this purpose. However, there are a few hurdles that need to be overcome to create a Windows executable. In this post, I will show how these hurdles can be overcome.

Below is a list of potential hurdles you might encounter when creating a binary, along with the measures to overcome these hurdles.

Undeclared implicit dependencies

Sometimes dependencies arise with libraries that can only be identified at runtime. For example, through the following code:

                with pd.ExcelWriter(output_file_path, engine='openpyxl') as writer:
                    best_solution.to_excel(writer, index=False, sheet_name='Solutions')

                    summary_df = pd.DataFrame(summary_data, columns=[
                        'max_invest',
                        'sum_missingSales',
                        'sum_refunds',
                        'current_ratio'
                    ])

These may not be recognized by your Python IDE and the scripts might still run locally in the IDE if you are not working with virtual environments, because the library is already installed globally.

The problem can be resolved by explicitly declaring the dependency in the import section:

import pandas as pd
import pulp
import easygui
import openpyxl # explicit declaration
from datetime import datetime

Suppress multiple Process Execution

In Windows, the problem sometimes occurs that the application is started multiple times when the app spawns child processes. This can be avoided with the following code:

# main program
if __name__ == "__main__":
    tk.Tk.report_callback_exception = log_uncaught_exceptions

    try:
        multiprocessing.freeze_support()  # suppress multiple main-processes on windows
        create_gui()
    except Exception as e:
        logging.exception("An error occurred in __main__.")

Missing external Binaries

When working with solvers, it often happens that the actual solvers are present as binary files on the host system. These are not included in the archive by PyInstaller, which results in the solver not being packaged into the executable, causing the executable to fail. Below is an explanation of how to configure the default solver used for solving MILP problems in the PuLP framework for deployment in an app using PyInstaller:

1. Identify the Solver-Binary
   Since I want to use the standard solver from PuLP for linear programs, I need to obtain the binary of the CBC solver. Depending on the system for which you want to create a binary, you need to find the correct binary. Since I want to produce a delivery for 64-bit Windows systems, I need the cbc.exe for 64-bit Windows systems:
   
   
   You can find this either in the venv directory of the IDE or in the system-wide package repository of the Python libraries.
   
2. Create a bin Directory in your Project Root
   Now create a ‘bin’ directory in the project root and copy the solver’s binary file into this directory.
   
   
   
3. Adapt solver invocation
   To call the solver, the script must now differentiate between whether it is being executed as a script or as a packaged archive. If the script is executed as a packaged .exe, the binary of the solver must first be unpacked and then called from the corresponding unpacking location. We add a function get_solver_path() to determine the solver’s location:
   
   
   See the script below defining the get_solver_path() function.
   
4. Pass the solver-Reference to the Solve-Call
   When dealing with MILPs, you need to define the linear model in your code. In the case of PuLP, we need to define a problem instance as shown below, then add the constraints and the objective function (not shown):
   
   
   
   After having modeled the MILP problem, you need to pass the solver-reference generated above to the problem instance as follows:
   
   
   
5. Pass the solver binary-location to the pyinstaller-Call
   As a last step, you need to provide the location of the binary solver-file to the pyinstallet call in order to create the distributable and executable binary for the destination platform (Win-64 in our case):
   
   
   
   If you want to suppres the cosnole-output, just add the –noconsole switch.

The following script shows the python-function necessary to identify the solver-binary location. This way it works in the development environment as well as in the packaged version of the python executable environment:

def get_solver_path():
    """
    determine location of the solver binary in either packaged version
    or in "ordinary execution mode".

    :return: path to the cbc binary
    """
    if getattr(sys, 'frozen', False):
        # If the application is run as a bundle, the PyInstaller bootloader
        # has set the sys.frozen attribute and sys._MEIPASS points to the
        # bundle directory.
        base_path = sys._MEIPASS
    else:
        base_path = os.path.abspath(".")

    solver_path = os.path.join(base_path, 'bin', 'cbc.exe')

    if not os.path.isfile(solver_path):
        # Create a temporary directory to store the binary if it doesn't exist
        temp_dir = tempfile.mkdtemp()
        solver_path = os.path.join(temp_dir, 'cbc.exe')
        with open(solver_path, 'wb') as f:
            f.write(pkgutil.get_data(__name__, 'bin/cbc.exe'))

    return solver_path

If the function is called for the first time from the packaged python executable, the solver binary will be extracted from the archive and copied to the executable root base location of the bundle directory. This way it is guaranteed, that the solver can be called as expected later.

This ensures that the application works correctly both as a script and when bundled.