Category: Software Development

Python try catch loop with increasing wait time


The idea here is if there is an occasional error that causes your whole procedure to abort, but it is just an intermittent glitch that will work just fine if you wait a bit, you can just trap for the error and retry a few times.

But each time you retry, the wait time increases, since you don’t exactly know when the issue will clear.  This is with Python 2.7.10.

import time

tries = 1
wait = [1, 5, 10, 15]
while tries <= 5:
        raise ValueError('Oops!')
        if tries < 5:
            print "Retry after " + str(wait[tries - 1]) + " seconds."
            time.sleep(wait[tries - 1])
        tries = tries + 1


Here’s the output:

Retry after 1 seconds.
Retry after 5 seconds.
Retry after 10 seconds.
Retry after 15 seconds.

In real life, the differences between the wait times are bigger.  I did 1 minute, 5 minutes, 30 minutes, and an hour.

Also of course you would have your real code inside the try/except.  The raise ValueError(‘Oops!’) is just to test the concept of the try/catch loop with increasing wait times.

If you know the exact exception, you can trap for it specifically, like below:

except ValueError as e:

where ValueError is substituted with the actual exception you are trapping for.

Getting SQL Replication Info Programmatically from Java

winterWisteria Wisteria leaves in winter.



SQL Server has a process called replication, whereby you can automatically copy records from one database to another, and keep them in-sync.  One use for this is to have one database as the transactional database, and another to report from, with the reporting database having indexes and constraints tailored more toward faster reporting (reads) than fast input.

Sometimes replication gets a little behind, and if you are running reports on the database that is a little behind, you might like to know.

The stored procedure sp_replcounters will tell you the count of transactions awaiting transfer to the distribution database (which lies between the transaction database and the destination database in a replication process).  You can call this stored procedure, but you want it to return the result set to Java.

One way to achieve this is to wrap your own stored  procedure around it, put the records into a temp table and query the temp table to return the result set.  This only works if you specify SET NOCOUNT  ON, otherwise the Java Persistence API (JPA) gets confused by the multiple queries and you don’t get the result set in your Java code.

CREATE PROCEDURE GetReplicationLatency


--This is important.  Results from multiple queries within stored procedures throw JPA off unless SET NOCOUNT ON is used.

declare @db varchar(50)
set @db = db_name()

create table #tempTable ([database] varchar(50), [replicated transactions] int, [replication rate trans/sec] int,
 [replication latency (sec)] decimal(16,6), replbeginlsn varchar(40), replnextlsn varchar(40))
insert into #tempTable
exec sp_replcounters
select * from #tempTable where [database] = @db

drop table #tempTable


Then your Java code looks like this:

private static final String REPLICATION_LATENCY_QUERY = "exec GetReplicationLatency ";Query query = getEntityManager().createNativeQuery(REPLICATION_LATENCY_QUERY);
Query query = getEntityManager().createNativeQuery(REPLICATION_LATENCY_QUERY);
List<Object[]> results = query.getResultList();



Hibernate Dynamic Update

ThorAndMe  That’s a pic of me with Thor.

I had a situation come up where every time a particular table was updated, an indexed view was also getting updated.  That view was intended to speed up a report, but it made the table update really horrible.  The view would only get updated if one particular column in the table was being updated.  That column never got updated through the application, because once the record was created that column’s input field got grayed out.  But Hibernate’s default behavior is to update all columns rather than to check which ones changed and only update the changed columns.

There is a performance trade-off.  Usually you probably want the default behavior.  But in this case, definitely not.

But Hibernate has an option to allow for updating only the columns that changed.  This option is called dynamic-update.  Different versions of Hibernate implement it differently.  In the example in this blog post, hibernate-annotations version 3.4.0.GA was the dependency brought in.

Because Hibernate’s org.hibernate.annotations.Entity class is not a replacement, but rather a supplementation of javax.persistence.Entity, you have to implement it like this:

import javax.persistence.Entity;
@org.hibernate.annotations.Entity(dynamicUpdate = true, selectBeforeUpdate = true)
@Table(name = "blah")
public class Blah...

The selectBeforeUpdate option is not required with the dynamicUpdate.  It basically states that before doing an update, do a select to make sure you have the most recent copy of the table, then the dynamicUpdate option says only update changed columns.

Creating a config file for MSI packaging software for an executable name that varies


The problem

Client wants our software packaged as an MSI (Microsoft Installer).  Found some software that will do this called exemsi.  Exemsi comes in a free mode with a GUI interface and a professional mode ($200 per machine) that can be called from a build process.

The professional mode relies on a configuration file.  The name of the executable file that you want to wrap in MSI is specified in the configuration file.

If the name of the executable file changes from release to release (for example, if it has a version number as part of the file name), the configuration file is supposed to have a way of finding the file based on wildcards and regular expressions.  I could not get this to work and had extreme difficulty reaching the author of the software.  He did reply to me once, but his suggestion did not solve the problem.

So, after much frustration, I decided to just make a little script that would read from a basic template configuration file, and write out nearly every line to a temporary configuration file for the current build.  Nearly every line because one line would have a substitution.  The line that specifies the executable name would be substituted for the installer file name of the current release.

I started writing this script as a DOS batch file.  I ran into trouble trying to make a loop with multiple command lines inside the loop.  I started thinking I’d scrap DOS and do it in Python, when my boss suggested Powershell.  Either one would serve.  The two machines on which this script would need to run already had Powershell installed, so that’s what won.

Besides, it afforded me a nice opportunity to learn how to use Powershell, having never used it before.

The solution

Here’s the Powershell script, which I called makeMsiWrapperConfig.ps1:

#To call from command line:
#  powershell -noexit "& ""path\makemsiwrapperconfig.ps1"" ""path\installer executable file name"" ""path\exemsi config template file"" ""path\resultant exemsi config file"""
#  where path is substituted with the actual path to the powershell file,
#  and installer executable file name is substituted with the actual lite installer file name,
#  and exemsi config template file is the name of the template configuration file for the exemsi software that wraps our installer executable in an MSI package,
#  and resultant exemsi config file is the output file with the correct executable name in it.
#  The outer set of quotation marks is so that powershell won't incorrectly parse the parameters if there are any spaces in the paths or filenames.
#  The next level of quotation marks is to escape the inner-most level of quotation marks so that once inside the script, the variable will be quoted.
#  powershell -noexit "& "".\makemsiwrapperconfig.ps1"" "".\target\app-123456.exe"" ""c:\resources\msiWrapper.xml"" "".\target\msiWrapper.xml"""  

$exeFile = $args[0]
$configTemplate = $args[1]
$resultantConfig = $args[2]
write-host "Creating configuration file for exemsi, the software which wraps the lite installer in an MSI package.  Parameters:"
write-host "  exe file to wrap - " $exeFile
write-host "  config template - " $configTemplate
write-host "  resultantconfig - " $resultantConfig
" " | set-content -path $resultantConfig
foreach($line in Get-Content $configTemplate) {
    $exportLine = $line
    if ($line -like '*<Executable*') {
        $exportLine = '<Executable FileName="' + $exeFile + '"/>'
    #write-host $exportLine
    $exportLine | add-content -path $resultantConfig


It just reads every line from the template exemsi configuration file, and when it finds the line that has “<Executable” in it, makes a substitution using the file name from the parameter passed in.  Then it spits out the lines to a tailored-to-the-current-release configuration file.

I may tweak this to customize other lines in the configuration file.  For example the version number is one of the things in the configuration file that would vary from release to release and that has its own tag.  So more work will likely need to be done but at least this proves the concept.

The next thing I needed to do was call this from the Apache Ant build.xml file.  Here’s that code (the names have been changed to remove references to proprietary software):

<!-- create MSI (Microsoft Installer) artifacts -->
<exec executable="powershell" >
    <arg line="-noexit"/>
    <arg line="&amp; c:\resources\makemsiwrapperconfig.ps1 .\target\app-123456.exe c:\resources\msiwrapper.xml .\target\msiwrapper.xml"/>
<exec executable="c:\resources\MsiWrapperBatch.exe">
    <arg value='config=.\target\msiWrapper.xml'/>

It works.  I would hope there are easier solutions than this.  But once I had asked my boss to purchase the exemsi software and spent some time learning it, I was kinda like a dog with a bone and determined to make this solution work.  And hey, I learned something about Powershell and how to call a Powershell script from an Apache Ant build.xml file in the process.