View unanswered posts | View active topics It is currently Thu May 21, 2026 12:53 am



Reply to topic  [ 7 posts ] 
 TWA file format 
Author Message
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post TWA file format
Is the TWA file format publicly known?

_________________
Suddenly you're Busted!


Sun Feb 26, 2012 2:12 pm
Profile WWW
Gameop

Joined: Wed Jun 07, 2006 2:00 am
Posts: 241
Location: Daytona Beach, Florida
Unread post Re: TWA file format
Mongoose wrote:
Is the TWA file format publicly known?



The NSA knows...


<cue spooky music>

_________________
-=Darkstarbase TWGS=-
http://www.darkstarbase.com
telnet://darkstarbase.no-ip.org:23
Like us on Facebook http://facebook.com/darkstarbase


Sun Feb 26, 2012 7:01 pm
Profile ICQ WWW
Site Admin
User avatar

Joined: Sun Dec 24, 2000 3:00 am
Posts: 3151
Location: USA
Unread post Re: TWA file format
I haven't made it public, but it's not a secret either. No effort has been made to keep it secret. It's not encrypted or anything.

This is just a rough overview, but basically, the file is made up of a series of files all packed together, then compressed. For each file, the data stored is

1) 4 bytes indicating the length of the file path.
2) N bytes storing the file path, where N is the length stored in 1.
3) 4 bytes indicating the length of the file.
4) M bytes storing the file itself, where M is the length stored in 3.

Once all of the files are packed together, this file is compressed with zlib compression.

Unpacking this file involves decompressing, then reading the file path, then reading the file data and storing it to the file path. Note that the file path uses some aliases so that it doesn't represent an absolute file path. For example, it may contain "[SERVERROOT]", which is substituted with the game's main TWGS directory, or "[GAMEDIR]", which is substituted with the game directory where the game is being imported.

There may also be a "[GAMETITLE]Some Title" entry for the file path. This is the name of the edit and is not followed by file data.

This hasn't changed for a very long time. It is possible to add and remove files from the TWA without impacting the import/export, so it's pretty versitile. When I add a new data file, it's automatically supported. It's possible that there may be some changes in the future to support a watermark so gameops can know that an edit is unmodified from its original form as released by the original author.

_________________
John Pritchett
EIS
---
Help fund the TradeWars websites! If you open a hosting account with A2 Hosting, the service EIS uses for all of its sites, EIS will earn credits toward its hosting bill.


Tue Feb 28, 2012 12:02 am
Profile WWW
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post Re: TWA file format
Neato. This seems to work. The first byte of the file path is a redundant indication of its length (evidently the names are Pascal strings), and the length fields are little-endian. The LittleEndianDataInputStream I'm using came from here.

I'm toying with the idea of incorporating a TWA importer into Weapon M, so the program can have complete data about things like planet types and gold alien rank names. 8)

Code:
package krum.weaponm.twareader;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.DataFormatException;
import java.util.zip.Inflater;

public class Test1 {

   public static void main(String[] args) throws IOException, DataFormatException {
      InputStream resource = Test1.class.getResourceAsStream("/Equilibrium.twa");
      byte[] compressed = new byte[1048576];      // 1 MiB
      int compressedSize = resource.read(compressed);
      System.out.printf("Read %d compressed bytes\n", compressedSize);
      
      byte[] uncompressed = new byte[16777216];   // 16 MiB
      Inflater inflater = new Inflater();
      inflater.setInput(compressed, 0, compressedSize);
      int uncompressedSize = inflater.inflate(uncompressed);
      System.out.printf("Inflated %d bytes\n", uncompressedSize);
      
      ByteArrayInputStream input = new ByteArrayInputStream(uncompressed, 0, uncompressedSize);
      LittleEndianDataInputStream data = new LittleEndianDataInputStream(input);
      
      System.out.println("Scanning data stream...");
      byte[] nameBuffer = new byte[1024];
      
      while(data.available() > 0) {
         int nameLen = data.readInt();
         data.read(nameBuffer, 0, nameLen);
         String name = new String(nameBuffer, 1, nameLen - 1);
         if(name.startsWith("[GAMETITLE]")) {
            System.out.printf("Title: %s\n", name);
         }
         else {
            int fileLen = data.readInt();
            data.skipBytes(fileLen);
            System.out.printf("File: %s (%d bytes)\n", name, fileLen);
         }
      }
      
      data.close();
   }
   
}

_________________
Suddenly you're Busted!


Thu Sep 27, 2012 1:55 am
Profile WWW
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWA file format
You should also support Trade Wars Export File Format (TWX):

http://www.swath.net/?page=twx

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Thu Sep 27, 2012 2:11 am
Profile ICQ YIM WWW
Commander
User avatar

Joined: Mon Oct 29, 2001 3:00 am
Posts: 1096
Location: Tucson, AZ
Unread post Re: TWA file format
I'm planning to support TWXv2.

_________________
Suddenly you're Busted!


Thu Sep 27, 2012 2:31 am
Profile WWW
Ambassador
User avatar

Joined: Wed Apr 20, 2011 1:19 pm
Posts: 2559
Location: Oklahoma City, OK 73170 US
Unread post Re: TWA file format
I think the TWX helper only supports TWX v1 format.

_________________
Regards,
Micro

Website: http://www.microblaster.net
TWGS2.20b/TW3.34: telnet://twgs.microblaster.net:2002

ICQ is Dead Jim! Join us on Discord:
https://discord.gg/zvEbArscMN


Thu Sep 27, 2012 2:38 am
Profile ICQ YIM WWW
Display posts from previous:  Sort by  
Reply to topic   [ 7 posts ] 

Who is online

Users browsing this forum: No registered users and 19 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by wSTSoftware.