www.ClassicTW.com
https://mail.black-squirrel.com/

TWA file format
https://mail.black-squirrel.com/viewtopic.php?f=14&t=32958
Page 1 of 1

Author:  Mongoose [ Sun Feb 26, 2012 2:12 pm ]
Post subject:  TWA file format

Is the TWA file format publicly known?

Author:  Darkstarbase [ Sun Feb 26, 2012 7:01 pm ]
Post subject:  Re: TWA file format

Mongoose wrote:
Is the TWA file format publicly known?



The NSA knows...


<cue spooky music>

Author:  John Pritchett [ Tue Feb 28, 2012 12:02 am ]
Post subject:  Re: TWA file format

I haven't made it public, but it's not a secret either. No effort has been made to keep it secret. It's not encrypted or anything.

This is just a rough overview, but basically, the file is made up of a series of files all packed together, then compressed. For each file, the data stored is

1) 4 bytes indicating the length of the file path.
2) N bytes storing the file path, where N is the length stored in 1.
3) 4 bytes indicating the length of the file.
4) M bytes storing the file itself, where M is the length stored in 3.

Once all of the files are packed together, this file is compressed with zlib compression.

Unpacking this file involves decompressing, then reading the file path, then reading the file data and storing it to the file path. Note that the file path uses some aliases so that it doesn't represent an absolute file path. For example, it may contain "[SERVERROOT]", which is substituted with the game's main TWGS directory, or "[GAMEDIR]", which is substituted with the game directory where the game is being imported.

There may also be a "[GAMETITLE]Some Title" entry for the file path. This is the name of the edit and is not followed by file data.

This hasn't changed for a very long time. It is possible to add and remove files from the TWA without impacting the import/export, so it's pretty versitile. When I add a new data file, it's automatically supported. It's possible that there may be some changes in the future to support a watermark so gameops can know that an edit is unmodified from its original form as released by the original author.

Author:  Mongoose [ Thu Sep 27, 2012 1:55 am ]
Post subject:  Re: TWA file format

Neato. This seems to work. The first byte of the file path is a redundant indication of its length (evidently the names are Pascal strings), and the length fields are little-endian. The LittleEndianDataInputStream I'm using came from here.

I'm toying with the idea of incorporating a TWA importer into Weapon M, so the program can have complete data about things like planet types and gold alien rank names. 8)

Code:
package krum.weaponm.twareader;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.DataFormatException;
import java.util.zip.Inflater;

public class Test1 {

   public static void main(String[] args) throws IOException, DataFormatException {
      InputStream resource = Test1.class.getResourceAsStream("/Equilibrium.twa");
      byte[] compressed = new byte[1048576];      // 1 MiB
      int compressedSize = resource.read(compressed);
      System.out.printf("Read %d compressed bytes\n", compressedSize);
      
      byte[] uncompressed = new byte[16777216];   // 16 MiB
      Inflater inflater = new Inflater();
      inflater.setInput(compressed, 0, compressedSize);
      int uncompressedSize = inflater.inflate(uncompressed);
      System.out.printf("Inflated %d bytes\n", uncompressedSize);
      
      ByteArrayInputStream input = new ByteArrayInputStream(uncompressed, 0, uncompressedSize);
      LittleEndianDataInputStream data = new LittleEndianDataInputStream(input);
      
      System.out.println("Scanning data stream...");
      byte[] nameBuffer = new byte[1024];
      
      while(data.available() > 0) {
         int nameLen = data.readInt();
         data.read(nameBuffer, 0, nameLen);
         String name = new String(nameBuffer, 1, nameLen - 1);
         if(name.startsWith("[GAMETITLE]")) {
            System.out.printf("Title: %s\n", name);
         }
         else {
            int fileLen = data.readInt();
            data.skipBytes(fileLen);
            System.out.printf("File: %s (%d bytes)\n", name, fileLen);
         }
      }
      
      data.close();
   }
   
}

Author:  Micro [ Thu Sep 27, 2012 2:11 am ]
Post subject:  Re: TWA file format

You should also support Trade Wars Export File Format (TWX):

http://www.swath.net/?page=twx

Author:  Mongoose [ Thu Sep 27, 2012 2:31 am ]
Post subject:  Re: TWA file format

I'm planning to support TWXv2.

Author:  Micro [ Thu Sep 27, 2012 2:38 am ]
Post subject:  Re: TWA file format

I think the TWX helper only supports TWX v1 format.

Page 1 of 1 All times are UTC - 5 hours
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/