Share Your TK Models!
Calling all authors! We'd love to see what you've done in TK, and share it with the greater TK community. Click
here to see models TK users have shared with us in the past.
All models should be sent to
todd.piefer@uts.us.com. Please check out the TK Model Share Guidelines for details.
TK Model Share GuidelinesWhen submitting a file for TK Model Share, please include the phrase "Model Share" in the email subject line.
Models should be properly documented, including a summary comment on the Comment Sheet. All variables on the variable sheet should include a description in the comment field. Additionally, you might consider including a separate DOC, PDF, XLS or other file to help document your model and zip them together into a compressed file.
If a model includes content in the MathLook Sheet, this information is stored in a separate file and its existence should be noted in the summary comment in case the files get separated. The path to the MathLook content file can be found at the bottom of the TK model file when viewed in a text editor. The path must be updated by each user who downloads the model. It might be best to avoid the use of MathLook content and simply zip a DOC or PDF with your model to display any diagrams or other associated graphics.
The TK model should not contain any "Included" files. Please merge any included files before submitting models.
You may also submit linked or packaged files produced using the TK-Excel Toolkit. If you submit a linked spreadsheet file that has not been packaged, be sure to submit the associated TK model(s) as well so that the link can be updated on each user's PC.
Labels: case studies, share your TK models, TK Solver, TK Solver models
Check Out These Success Stories!
Case studies and examples are a good thing. Sometimes, though, it's nice to read something a little less formal - say, an interview or first-hand account from a fellow TK user.
Click
here to read how an award winning petroleum engineer uses TK Solver.
Click
here to access a library of success stories.
Labels: case studies, customer success stories, TK Solver, TK Solver models
Have you visited the TK Library lately?
The hundreds of tools in the TK Solver Library can save you hours of programming time. Many of the tech calls I receive regarding application development with TK are answered with the use of one or more TK Library functions. It's good to stay acquainted with all that's in there. Here’s a quick tour of some of the most popular Library tools and some of the more recent additions.
Bilinear Interpolation
The inpol2 function is found under Utilities -- Model Solving. It can be used to convert tables of data (families of curves) into two-way look-up tables with interpolation.
Calendar Functions
These are found under Utilities – Miscellaneous. There are six functions for computing days between dates, a useful collection for financial calculations.
Graphics Tools
The “pickit” tools are found under Utilities – Graphics. This collection of functions generates plot coordinates for lines, boxes, circles, and other shapes and can rotate, translate, mirror, and otherwise transform the coordinates. There are also 34 functions found in the Mathematics – Geometry folder for doing geometry calculations frequently required for CAD. For example, there are various equations for lines, angles, points of intersection, rules for collinearity and parallelism and a function for determining the coordinates of the common tangent of two circles.
Unit Sheets
Perhaps the most commonly used time-savers in the TK Library, these unit sheets – arranged by discipline – can be merged into your TK models to allow for a variety of different display units for your variables and lists. These unit sheets are found under Utilities – Units and Conversions.
Fluids and Mixtures
The Utilities – Fluid Properties folder includes tools for calculations involving combustion, psychrometric charts and latent heat of vaporization but the Fluids and Mixtures file is probably the most frequently referenced. It summarizes the results from all the built-in fluid property functions.
Root Finders
The tools in the Mathematics – Roots of Equations folder are a must for TK application developers. These functions can be used instead of the built-in iterative solver to solve sub-tasks within an application such as generating plot data. The most recent addition to this folder is a tool called mbisect which hunts for and returns multiple roots of functions.
Functions of Complex Numbers
The 18 comfun tools under Mathematics – Complex Numbers compute functions of complex numbers such as square root, log, and a variety of trig functions.
Optimization
Similar to the root-finders, these tools can be used to process sub-tasks within TK models, freeing the Optimizer for use with the complete model. A very popular routine is the Marqoptn function based on the Marquardt algorithm for nonlinear optimization with numerical differentiation of the objective function with respect to the decision variables.
Random Number Generation
A recent addition to the Statistics – Distributions and Random Numbers folder provides the ability to generate correlated multivariate random values. This can be vital in performing Monte Carlo simulations where it is known that two or more input variables in a model are not independent.
The TK Library now includes random number generators for 33 different probability distributions, from Beta to Zipf-Mandelbrot. It also includes routines for fitting distribution parameters to lists of values.
Financial Modeling Tools
The Finance folder includes the very popular mortgage calculator and a few other, more sophisticated tools such as the Black-Scholes options pricing model.
Labels: TK Solver, TK Solver models
TK Solver Models Natural Disaster Outcomes for the Insurance Industry in South Africa
Adi Hazan is a “fixer.” A programmer of last resort, the guy people call when all other efforts have fallen through. As he describes it, “I take on projects where other people have failed or where there is a crisis . . . in general, I’m the last line of defense.” And it’s no wonder. Hazan sold his first program at age 12, and to this day is a voracious learner, blazing through 1000+ pages a week on “absolutely anything.”
One of Hazan’s most recent programming challenges arrived 9 months ago, when he was approached by an actuary who hired him to solve some outstanding problems in mathematical modeling of natural disasters. A legislative deadline loomed: The South African government had made modeling a legal requirement for the insurance industry starting in 2008. When Hazan was approached, no one had been able to produce a workable model. After weeks of testing actuarial programs without success, he broadened his search and came across TK Solver, and immediately recognized the potential.
So what does the program do? Hazan explains: “The program tries to predict what the maximum damage from earthquakes could be so that insurance companies can keep the correct money reserves for such an emergency. A more detailed explanation is that the system clusters the historical data into regions, and then models the frequency and magnitude for the regions using 30 different statistical curves in TK, optimizing the curves to fit the data. Once the best fit is chosen from the 30 curves, a simulation of 10,000 years is done using random inputs to see what size of damage one can expect every 250 years on the client’s specific set of insurance policies.”
TK’s speed, power and accuracy were critical to Hazan’s work. On TK’s speed, he states, “It’s vital to note that any series of calculations done in TK is at least 10 times faster than Excel. This is not trivial when you consider a 10 hour process coming down to an hour.” As to TK’s power, Hazan comments, “We were able to solve problems using built-in functions that cannot be done on any platform that costs less than $150,000. Specifically, we were able to fit a curve of rounded, discrete values to a mean and standard deviation–what this means is that we can round and then optimize, not optimize and then round and lose accuracy. This is impossible on normal algorithms or the Frontline Solver for Excel.” Finally, accuracy was imperative to Hazan’s program, and TK delivered. He remarks, “A number of the rarely-used functions in Excel simply gave us wrong answers in comparison. When the TK answers were wrong we were able to adjust the functions, whereas in Excel it was ‘take-it-or-leave-it.’”
Hazan also cites TK’s support as “unparalleled,” in particular the expertise Todd and Mohan provided, on both complex statistical issues and non-mathematical technical support. As he puts it, “Imagine a call center getting a call asking how to ‘expand the Marquardt optimization algorithm so that it can accommodate discrete values and still move to the local minimum of multi-variable curves.’”
As for response to the program Hazan reports that it has been “excellent.” TK’s versatility prevented him from having to side-step any mathematical issues, giving Hazan the ability to build something “more detailed and accurate than anything else on the market.”
Labels: case studies, natural disaster modeling, TK Solver, TK Solver models
Math Models and Monte Carlo Simulation
It's always nice to know your odds of success before taking on any challenge. In some cases, such as card games, the odds can be computed from the information known to you. In other cases, odds are based on assumptions and/or past performance. This is true for things like sporting events and performance of financial products. Engineering design can also involve probabilities. Design engineers doing tolerance stack calculations are determining the odds that things will fit in a given space. In many cases, the required calculations are prohibitively complex. That is when Monte Carlo simulation becomes so appealing.
Monte Carlo simulation involves running a math model repeatedly using "random" numbers from assumed distributions as the inputs. Then you observe the distributions of the output variables and use them to predict the probabilities of seeing values within particular ranges.
Simulation in TK Solver has the same prerequisites as List Solving, plus a few more. The model must have input and output variables on the variable sheet and the model must solve using the direct solver or with automatic iteration (i.e. - without having to manually provide guess values).
Next, you need to determine if any of the simulation inputs are correlated. If two variables are correlated, they tend to vary together. If any input variables are correlated, the random values used as inputs must be generated with that in mind. This can have a significant impact on the simulation results, as the results will show greater variability if no correlations are assumed.
TK Solver's list fill facility can be used to generate lists from a variety of distributions but it assumes that the variables are uncorrelated. The TK Library includes additional procedures for generating independent lists of random numbers.
We recently developed a procedure (let's call it "rancorr" for now) for generating correlated sets of lists. Rancorr provides a table for specifying correlations between the input variables. The correlation matrix for the input variables defaults to 0’s off the diagonal, indicating that the variables are independent. If non-zero correlations between any or all of the variables are to be assumed, edit the correlation matrix and the simulation will account for it. Correlations can vary from -1 to 1. The correlation matrix is symmetric, so if you enter a correlation in row m and column n of the matrix, the same value is automatically entered in row n, column m. It is possible to construct a correlation matrix that is mathematically infeasible. The procedure checks and reports on that condition.
Rancorr includes another table for specifying the distribution parameters for each variable. Here is the list of distributions:
Bernoulli, Beta, Binomial, Cauchy, Erlang, Exponential, Extreme Value, Gamma, Half Normal, Hypergeometric, Kumaraswamy, Log Series, Logistic, Lognormal, Negative Binomial, Normal, Poisson, Power, Rayleigh, Rectangle, Semicircle, Skellam, Trapezoid, Triangle, Truncated Normal, Uniform, Weibull, Weibull Series, Yule Simon, and Zipf Mandelbrot.
The TK Library includes many routines for fitting various probability distributions to lists. By fitting, we mean that they determine the parameters that cause a particular distribution to match a list of values as closely as possible. It is easy to launch a second TK, load a Library routine and then copy and paste an output list from a simulation run into the fitting routine.
That method assumes that you have some notion as to which distributions might fit the data. Each library routine does its best to determine the parameters of a specific distribution but does not compare the results to determine which fit best. We recently developed a TK routine called "fitall" which does just that. Fitall can run through a series of over 30 different distributions and determine the parameters of the one which fits a list the best.
Fitall processes each of the output lists, providing a frequency histogram, probability plots and descriptive statistics for a visual and numerical summary. The histogram also includes an overlay of the best fitting theoretical distribution. The descriptive statistics for the simulated values include:
Max/Min – Maximum and minimum values
Mean – Arithmetic mean
Standard Deviation – A measure of the variability
Mean +/- 3SD – The arithmetic mean plus (or minus) three standard deviations is often used as a practical estimate for the expected range.
Skewness – Used to compare the distribution with a normal distribution, skewness is an indication of the symmetry of the results about the mean value. The skewness for a normal distribution is zero, and any symmetric data should have skewness near zero. Negative values for the skewness indicate data that are skewed left and positive values indicate data that are skewed right. By skewed left, we mean that the left tail is heavier than the right tail.
Kurtosis - Used to compare the distribution with a normal distribution, kurtosis is an indication of the “peakedness” of the resulting distribution. Positive kurtosis indicates a "peaked" distribution and negative kurtosis indicates a "flat" distribution.
If the data contains any values less than or equal to zero, some of the distributions are eliminated. A chi-square test of the frequencies is done. Anderson-Darling and Kolmogorov-Smirnov test statistics are computed. The correlation between the data and the theoretical data is computed. The theory is that the best fitting distribution should have the smallest Chi-Square, Anderson-Darling and Kolmogorov-Smirnov statistics and the largest correlation coefficient.
The results from each distribution are ranked and the ranks summed to form a score. The distribution with the lowest score is selected. Tables summarize the results from each distribution tested along with the ranked scores.
Plots are available for each distribution. A plot of the best-fitting cumulative distribution is shown along with the actual data. A bar chart shows the observed and expected frequencies for five or more intervals.
Once you have the distribution parameters, you can compute the probability of observing values in any particular range. In effect, you now know the odds. The process only takes a minute or two so if you don't like the odds, you can tweak your math model and re-run the simulation until you're satisfied.
We are in the process of creating a Simulaton Wizard to automate the steps involved in using rancorr and fitall. Meanwhile, we are looking for individuals interested in testing these new tools. Please contact me if you are interested (todd@uts.com). You read this far... why not give them a try?!
Labels: monte carlo, TK Solver, TK Solver models
