A Heretical Question - Pitfall Ahead?

LAST EDITED ON Nov-17-04 AT 10:43 AM (EST)

Hi guys,- First, I am aware of the other thread going on, related to problems with global/tsection build 27.
- Second, I am a great fan of all the glorious add-on sets for tracks and routes and do make extesive use of them.
- Third, although no one else has stated that yet, I have the uncomfortable feeling that we are close to a critical point here - namely with the sheer size the global/tsection file is growing to.
- Fourth, when referring to size, I don't think of the "user friendliness" when searching some specific piece in there. I think of file size and memory per se.I want to explain below why I ask this heretical question, but in a nutshell:
"Is global/tsection.dat simply about to become too big for MSTS to work properly?" Over the past few months, I started developping "out-of-memory" errors with my install of MSTS, i.e. when running an activity, MSTS would suddenly fail to load any object on the next tile I approached. At first, it only happened on large routes, when several large tiles were involved, and in activities with quite some traffic. I thus didn't feel too alarmed about it, quit, reloaded from the last save and all was fine for the rest of the run.Since then, the situation has slowly deteriorated, despite the fact that my desktop is not that low-ended (1GB of RAM, GeForce etc.). I installed new roads 3.2. and global/tsection.dat build 27 yesterday. No problem with loading the sim thereafter, but... BUT... While running a mid-sized route today (.rdb stands at 2187KB, .tdb at 3421KB), on which I never ever have had any memory issues before, I got the dreaded out-of-memory errors when approaching a tile that was dense with scenery, but by far not (yet) the densed one of the route. Now we all know that MSTS loads a number of files permanently into memory whenever we run an acitivity or explore a route. These are the various root files of the route (.rdb, .rit, .tdb, .tit, plus all .dat files needed like sigscr.dat, sigcfg.dat, carspawn.dat etc). Apart from these, MSTS also needs to load a couple of global .dat files, such as camcfg.dat, soundcfg.dat, and last but not least the global/tsection.datAll these files need to be in MSTS' memory permanently, as well as the files associated with an activity and its traffic (i.e. all .srv, .trf, .pat files). The only thing that is juggled in and out of memory are the .w and .t files which are only displayed as needed (sorry to the experts that already knew that for explaining it all again).The original, unmodified tsection.dat KuJu had in mind when programming the sim and assessing the memory needed was 156KB. The latest global/tsection.dat build 27 is roughly 20x (twenty times) larger than that, at 3300KB. Go figure...I might be crying "wolf" here prematurely, but I currently feel pretty much like we were approaching a pitfall:
-> out-of-memory errors became more and more frequent,
-> I regularily updated whenever a new global/tsection.dat came out,
-> immediately after installing the latest one, I start getting out-of-memory errors within 15 minutes of exploring a route which never gave me this error before, not even in complex activities. Furthermore, the fact that only some users tend getting the problems mentioned in thread http://www.train-sim.com/dcforum/DCForumID6/12712.html while others (still) do fine, could also be explained by the theory that file size is becoming a problem, simply because the overall memory available varies from user to user (MSTS start-up factors like number of routes, consists etc. may well also play a role here). Final remark: I am aware that RouteRiter will remove all unused .s files from the global/tsection.dat. Will soon try if that cures my problem, but would greatly appreciate your input here.Lukas a.k.a Swissie

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

RE: A Heretical Question - Pitfall Ahead?, OTTODAD, Nov-17-04, 11:30 AM, (1) RE: A Heretical Question - Pitfall Ahead?, buttercup, Nov-17-04, 11:39 AM, (2) RE: A Heretical Question - Pitfall Ahead?, Steven, Nov-17-04, 05:19 PM, (3) RE: A Heretical Question - Pitfall Ahead?, henry3, Nov-18-04, 12:04 PM, (4) RE: A Heretical Question - Pitfall Ahead?, buttercup, Nov-18-04, 12:35 PM, (5) RE: A Heretical Question - Pitfall Ahead?, robertreedy, Nov-18-04, 05:50 PM, (6) RE: A Heretical Question - Pitfall Ahead?, henry3, Nov-19-04, 12:53 PM, (7) RE: A Heretical Question - Pitfall Ahead?, muskokaandtahoe, Nov-19-04, 08:58 PM, (8) RE: A Heretical Question - Pitfall Ahead?, muskokaandtahoe, Nov-19-04, 09:01 PM, (9)

Conferences | Topics | Previous Topic | Next Topic

Hi Lukas !You are touching on something I have become suspicious about recently when the subject of Coupler Breaks was raised again, which some have and some do not.When a train starts, MSTS is keeping track of it's movements as it drives along the route, storing some sort of log in the computer's memory, calculating distances between Nodes, etc ? The longer the journey the more data being stored ?If available memory becomes scarce and swapping some data to Virtual Ram on a hard drive is inefficient could this be one of the reasons for problems to appear with some and not others ?The MARIAS-KOOTENAI's Tracks and Road Databases are a combined 10450 Kb. Add to this 3300 Kb for the Tsection 00027 and other files MSTS needs permanently in memory, approaching "Critical Mass" could be a possibility on some computers ?Take Care, O t t o.

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

If you want to have a separate global tsection for each route you can automate the process with Train Store.My system has an installation as follows: Routes: 275
Activities: 2,960
Services: 10,115
Paths: 10,052
Consists: 7,609
Stock Items:
Engines: 3,957
Wagons: 7,398
Total: 11,355One of the routes requires a special global tsection. When I un-store this route the global tsection is automatically swapped for the required one.Everything functions smoothly.Regards,Eric

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

I will throw my 2 cents in for whats its worth.Master Tsection
The tsection.dat file is big and getting bigger. In terms of definition space there is still a long way to go before being maxed out. Tried a little experiment using Martyn T. Griffin's Tsect.exe program. I ran build 27 through the program using the "terse" option which converts it into single line entries and eleminates the "INFO/SKIP" sections. The result was reducing the original filesize of 3,300KB down to 2,711KB. Thats 589KB of comments, spaces, tabs, etc that make it more readable, but could be eliminated.I think a "standardized tsection" is still needed to hopefully eliminate potential conflicts between shape names which all go into the GLOBAL\SHAPES folder. But this should be used as the "base" file for other solutions.Route-Riter
Route Riter already has a "Make Tsection" option which eliminates entries that you do not have a shape for in the GLOBAL\SHAPES folder. Some more space can be reclaimed by also having the output tsection in terse format.Train-Store
Train-Store has the ability to use a separate tsection.dat file for each route. This is great, a possibility here would be a "Make Route Tsection" option that would eliminate any unused section/shape entries that are not used by the route in question. But like Route Riter still have the "master tsection" as a base to draw from.
Route Riter has a start on this with the ability to display which track/road sections are used by a route, just need to carry it a bit further.StevenStandardized \global\tsection.dat file now at build #27.

LMRR - Lower Mainland Railways (of BC)
Track: 1080.12 km
Roads: 1228.27 km

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

LAST EDITED ON Nov-18-04 AT 12:09 PM (EST)

Lukas,If you have the Lehigh Valley route installed, check and see if you can run on it. It has one of the largest tdb's that I'm aware of (over 5,000KB). I didn't check the rdb, but I imagine that it's fairly sizeable as well. And the route has a number of fairly dense tiles.I think that memory problems should show up there, if anywhere.I have a personal route that I've been messing with from time to time. Recently I have been getting error messages concerning the freeway sections that I installed earlier. I was hoping that your message might have given me a lead on solving the problem, but I don't think so. The tdb and rdb for the route are only in the 2,000KB range, and I can run the Lehigh Valley just fine.Henry

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

I have for time to time had some problems with some files not loading and have found that it can be solved (not always) by adding it to the global and/or appropriate route folder. I am wondering when the sim is under "stress" that it looks in the wrong place for the particular file. Also the length of the files name (or path string) seems to have some effect as I have had some success in shortening the name of the problem file (also making the appropriate changes to the files that call it).Regards,Eric

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

>If you have the Lehigh Valley route installed, check and see
>if you can run on it. It has one of the largest tdb's that
>I'm aware of (over 5,000KB). I didn't check the rdb, but I
>imagine that it's fairly sizeable as well. And the route
>has a number of fairly dense tiles.
>
>I think that memory problems should show up there, if
>anywhere.
>
Henry, I'm not Lukas, but have both LeHigh Valley and ADFRR installed. ADFRR's tdb is 6656kb and rdb is 2180, slighty larger than LV. I have 2gig of memory, new fast computer, and can run ADFRR all day long with few problems, but only last in LeHigh for 15 or 20 minutes before it plops to desktop. Go back in for 15-20 minutes and it's desktop again. I think maybe the 'denser tiles' might have something to do with it along with the increased size of the standardized tsection bat, but am not really sure. Maybe even some larger textures, but don't know and am kind of grasping at straws here.At any rate, I've just trimmed the tsection down with RR and am going to go back in and see what happens. Couldn't hurt anyway.btw, Thanks Lucas for bringing this up as it may be a contributing factor after everything is said and done.RobertR .. ADF-M&RM at http://www.adfrr.com

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

Robert - Okay, that's disturbing. From the looks of your specs you should be among the last person to have problems with any routes. I have never experienced that behavior with Lehigh Valley, even when I had frame rates down in single-digits.Eric - I added the freeway sections to the shape and texture subfolders of the route as you suggested, and now everything looks like its working fine. I wish I knew *why* it was working, though.Henry

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

LAST EDITED ON Nov-19-04 AT 08:58 PM (EST)

>out-of-memory errors became more and more frequent, Is there a known limit of how much Ram MSTS will make use of or can it use everything the machine has to offer?Dave NelsonWP mainline, Oakland-Stockton: 100% complete;
SP mainline, Oakland-Stockton: 20% complete;
Aug 1950
<<new .sig under construction>>

Alert | IP

Printer-friendly page | Edit | Reply | Reply With Quote | Top

>"Is global/tsection.dat simply about to become too big for MSTS to work properly?" I wonder if there is practical way to redirect the path from /global to some specific-to-the-route directories that would come with the route. Sort of along the lines of how paths to sounds are changed.Dave NelsonWP mainline, Oakland-Stockton: 100% complete;
SP mainline, Oakland-Stockton: 20% complete;
Aug 1950
<<new .sig under construction>>

Alert | IP

[ Home | MSTS Forum | MSTS FAQ | How To... | File Library ] Copyright © 2004 by Nels Anderson. All Rights Reserved.