• Tossing of files with Spaces on Windows vs. Linux

    From Alisha Stutz@2:301/113 to g00r00 on Monday, January 10, 2022 10:25:42
    Hi G00r00,

    I always have problems receiving and processing files with TIC and
    attachments (tossing) if the attachment or the TIC file has a space in the name. This does not cause any problems for Windows users because they can handle spaces, but it does for us Linux and Raspberry Pi users with a Linux.

    The files then remain in the echomail/in/ directory as data rubbish and generate errors in the mutil log because they cannot be processed.

    + Jan 09 09:24:49 File MyGUI v1.0.2.26.zip Area FSX_MUTL From 21:1/100
    ! Jan 09 09:24:49 Unknown inbound file MyGUI
    + Jan 09 09:25:37 File MyGUI v1.0.2.26.zip Area FSX_MUTL From 21:1/100
    ! Jan 09 09:25:37 Unknown inbound file MyGUI
    + Jan 09 19:05:36 File MyGUI v1.0.2.26.zip Area FSX_MUTL From 21:1/100
    ! Jan 09 19:05:36 Unknown inbound file MyGUI

    Is there a parameter or any other way to solve this problem? I have to find
    out each time who sent it and where it belongs and then do a manual cleanup. Especially if it is sent 3 times like in the other example.

    Best wishes and a happy new year even if late
    Alisha
    --- Mystic BBS v1.12 A46 2020/08/26 (Linux/64)
    * Origin: swissIRC BBS (2:301/113)
  • From Shaun Buzza@1:229/110 to Alisha Stutz on Monday, January 10, 2022 04:55:11
    Hi G00r00,

    I always have problems receiving and processing files with TIC and attachments (tossing) if the attachment or the TIC file has a space in
    the name. This does not cause any problems for Windows users because
    they can handle spaces, but it does for us Linux and Raspberry Pi users with a Linux.


    I can confirm that this is a problem on PiBBS, which, as you might guess, is running on a Pi.

    McDoob
    SysOp, PiBBS
    pibbs.sytes.net

    --- Mystic BBS v1.12 A46 2020/08/26 (Raspberry Pi/32)
    * Origin: PiBBS (1:229/110)
  • From mark lewis@1:3634/12.73 to Alisha Stutz on Monday, January 10, 2022 06:05:44
    On 2022 Jan 10 10:25:42, you wrote to g00r00:

    I always have problems receiving and processing files with TIC and attachments (tossing) if the attachment or the TIC file has a space in the name. This does not cause any problems for Windows users because they can handle spaces, but it does for us Linux and Raspberry Pi users with a Linux.

    for full compatibility, all files transferred with TICs should use the standard 8.3 naming convention... LFNs are not supported by the majority of TIC processors for numerous reasons...

    )\/(ark

    "The soul of a small kitten in the body of a mighty dragon. Look on my majesty, ye mighty, and despair! Or bring me catnip. Your choice. Oooh, a shiny thing!"
    ... Don't cook tonight - starve a rat today!
    ---
    * Origin: (1:3634/12.73)
  • From Alisha Stutz@2:301/113 to mark lewis on Monday, January 10, 2022 12:26:04
    Hi mark,


    for full compatibility, all files transferred with TICs should use the standard 8.3 naming convention... LFNs are not supported by the majority of TIC processors for numerous reasons...


    Probably but why MysticBBS with Windows do not have trouble? Only the Linux Based MysticBBS run always in trouble with spaces in the Filename.Probably g00r00 can make a fix witch remove or replace spaces by _ bevor tossing this out that Linux Sysops can be part of the Party without reverse engineering
    all the times. I know it's retro but ther shut be a solutin in the 21th
    century to support us *nix user as well.

    Br
    Alisha Manuela
    --- Mystic BBS v1.12 A46 2020/08/26 (Linux/64)
    * Origin: swissIRC BBS (2:301/113)
  • From Shaun Buzza@1:229/110 to Alisha Stutz on Monday, January 10, 2022 07:15:31
    Probably but why MysticBBS with Windows do not have trouble? Only the Linux Based MysticBBS run always in trouble with spaces in the
    Filename.

    Linux file system is very different from Windows. So different, that Windows cannot even see Linux files in the same computer.

    With Mystic, this causes a lot of little problems, and a few big problems.
    You just found one of these problems.

    I know it's retro but ther
    shut be a solutin in the 21th century to support us *nix user as well.

    Again, Windows and *nix are very different in how they use files. There is no easy solution to that.

    McDoob
    SysOp, PiBBS
    pibbs.sytes.net

    --- Mystic BBS v1.12 A46 2020/08/26 (Raspberry Pi/32)
    * Origin: PiBBS (1:229/110)
  • From Alisha Stutz@2:301/113 to Shaun Buzza on Monday, January 10, 2022 17:36:54
    Hi Shaun,


    Again, Windows and *nix are very different in how they use files. There
    is no easy solution to that.


    Yes of course, and well know but with Software is it possibel to write a extension bevor tossing to replace spaces by point "." or underline "_" and then the Problem is solved. Okay the file has a divrent Name with aditional
    "." or "_" but then Thossing around FTN Style Networks will be functionaly as well in the 21th century.

    That's why i hope that G00r00 kann implement a feature like this in A47sp1
    with a possible backport to A46sp1.

    Best regards

    Alisha Manuela
    --- Mystic BBS v1.12 A46 2020/08/26 (Linux/64)
    * Origin: swissIRC BBS (2:301/113)
  • From g00r00@1:129/215 to Alisha Stutz on Monday, January 10, 2022 11:28:34
    I always have problems receiving and processing files with TIC and attachments (tossing) if the attachment or the TIC file has a space in

    Thanks for letting me know!

    I believe this may have been fixed quite a long time ago but I could be wrong.

    You would need to upgrade to A47 to see if its fixed and if it is still an issue then I will create a test here and get it fixed up! You can find the latest version at mysticbbs.com.

    ... Press SPACEBAR once to abort, or twice to save changes

    --- Mystic BBS v1.12 A48 2022/01/07 (Windows/32)
    * Origin: Sector 7 * Mystic WHQ (1:129/215)
  • From g00r00@1:129/215 to Alisha Stutz on Monday, January 10, 2022 11:53:33
    I always have problems receiving and processing files with TIC and attachments (tossing) if the attachment or the TIC file has a space in
    the name. This does not cause any problems for Windows users because
    they can handle spaces, but it does for us Linux and Raspberry Pi users with a Linux.

    Another thing to consider here is that in Unix systems filenames are case sensitive. So for example:

    + Jan 09 09:25:37 File MyGUI v1.0.2.26.zip Area FSX_MUTL From 21:1/100

    If your .TIC has "MyGUI v1.0.2" as the filename but the actual file is "mygui v1.0.2" in lower case, then those are not the same files in Unix. This could be a reason why it would work in Windows (which is not case sensitive) but not in Linux.

    If we determine this is your issue, then maybe what I can do is add a new option that you can turn on to make that TIC filename check case insensitive in Unix.

    ... No honey, I can't eat with the family. My computer gets lonely!

    --- Mystic BBS v1.12 A48 2022/01/07 (Windows/32)
    * Origin: Sector 7 * Mystic WHQ (1:129/215)
  • From Alisha Stutz@2:301/113 to g00r00 on Monday, January 10, 2022 18:23:08
    Hi G00r00

    I know I need to update to the A47 version, but I have too many things on my plate at the moment, and I am also unable to work due to massive disc
    problems. I can't sit in front of the PC non-stop and do everything because I want to do a clean installation of the A47 with data transfer to /mysticbbs
    and no longer to /home/mystic/mysticbbs/..... There are a few things that
    need to be done, but it's not going so quickly with the disc problems.

    I always have problems receiving and processing files with TIC and attachments (tossing) if the attachment or the TIC file has a space i the name. This does not cause any problems for Windows users because they can handle spaces, but it does for us Linux and Raspberry Pi use with a Linux.

    Another thing to consider here is that in Unix systems filenames are case sensitive. So for example:

    Yes, I realise that the addon/option would have to harmonise the filenames
    all to lower case and without spaces that are previously replaced by underscores "_" before sending these files via the FTN mailer with a TIC file to it.

    The other question in general is whether it doesn't also make sense that file uploads are generally harmonised and stored on the hard disk in lower case
    and without spaces which are replaced by underscores "_". Then the problem would always be solved in any case.


    + Jan 09 09:25:37 File MyGUI v1.0.2.26.zip Area FSX_MUTL From 21:1/10

    If your .TIC has "MyGUI v1.0.2" as the filename but the actual file is "mygui v1.0.2" in lower case, then those are not the same files in Unix. This could be a reason why it would work in Windows (which is not case sensitive) but not in Linux.

    If we determine this is your issue, then maybe what I can do is add a new option that you can turn on to make that TIC filename check case insensitive in Unix.


    As far as I know, A47 users are also affected by the same problem, so it
    seems to me that this problem is not fixed yet. As I said, it would be best
    to have a parameter that you can switch on/off. So that the religion of the 20th century does not collide with the religion of the 21st century. There
    are different views on what needs to be solved where and when 8.3 or
    harmonised long file names.

    Best regards

    Alisha Manuela
    --- Mystic BBS v1.12 A46 2020/08/26 (Linux/64)
    * Origin: swissIRC BBS (2:301/113)
  • From g00r00@1:129/215 to Alisha Stutz on Monday, January 10, 2022 15:01:40
    I know I need to update to the A47 version, but I have too many things
    on my plate at the moment, and I am also unable to work due to massive

    I understand but unfortunately I cannot fix something in a version I released 2 years ago without a time machine, so there is little I can do for you without upgrading! ;)

    Yes, I realise that the addon/option would have to harmonise the
    filenames all to lower case and without spaces that are previously replaced by underscores "_" before sending these files via the FTN
    mailer with a TIC file to it.

    Linux allows spaces in filenames so the underscores are not needed. :)

    I believe what you've shown was just a logging bug in A46 that was fixed long ago and that it should still work even without upgrading, but I could be remembering it wrong.

    But to reiterate: Most likely the filename in the .TIC file does not match the case of the actual file on disk. In Linux they must be exact matches. If your TIC says TEST.ZIP and you have test.zip on your disk, it will not toss.

    In A48 there is now an option to allow case insensitive TIC tossing and it defaults to being enabled so it will just work! :)

    As far as I know, A47 users are also affected by the same problem, so it seems to me that this problem is not fixed yet. As I said, it would be

    I set up a test today to try this in Linux over my lunch break and it works for me.

    I think more than anything the filenames being case sensitive in Linux throws people off but that will be a thing of the past in A48 with the ignore_case option enabled.

    Here is a paste from the log of a BBS installed on Ubuntu so you can see it working for me, no underscores etc required.

    ----------------- MUTIL v1.12 A48 2022/01/10 Mon, Jan 10 2022 (loglevel 2)
    + Jan 10 11:58:50 Startup using mutil.ini
    + Jan 10 11:58:50 Process: Toss FDN/TIC Files
    + Jan 10 11:58:50 Waiting for BUSY nodes
    + Jan 10 11:58:50 Tossing test.tic
    + Jan 10 11:58:50 File My Filename.txt Area TESTFILES From 1:154/10
    + Jan 10 11:58:50 Created area TESTFILES
    + Jan 10 11:58:50 Added to "Test Files for import"
    + Jan 10 11:58:50 Scanning Hatches
    + Jan 10 11:58:50 Results: 1 import, 0 toss, 0 hatch, 0 bad in 0.02s
    + Jan 10 11:58:50 Shutdown Normal (0)

    --- Mystic BBS v1.12 A48 2022/01/07 (Windows/32)
    * Origin: Sector 7 * Mystic WHQ (1:129/215)
  • From Paul Hayton@3:770/100 to g00r00 on Tuesday, January 11, 2022 16:12:06
    On 10 Jan 2022 at 03:01p, g00r00 pondered and said...

    But to reiterate: Most likely the filename in the .TIC file does not
    match the case of the actual file on disk. In Linux they must be exact matches. If your TIC says TEST.ZIP and you have test.zip on your disk,
    it will not toss.

    Hi there :)

    Just adding to this thread (which I hope well help you debug stuff)

    I had hatched a file from Agency running Mystic BBS v1.12 A47 Linux/64 Compiled 2021/11/06 07:08:58 so not the final version of A47

    A file was uploaded to Agency by a uuser and was called

    MyGUI v1.0.2.26.zip

    There was a space between MyGUI and the v1. but I didn't think anything
    of it. I know better now :)

    I'd moved using sysop file editor tools to the Mystic Utils base then
    hatched it from Agency to fsxNet.

    Logs show


    ----------------- MUTIL v1.12 A47 2021/11/06 Sun, Jan 09 2022 (loglevel 3)
    + Jan 09 17:09:35 Startup using mailin.ini
    - Jan 09 17:09:35 EXEC FileToss
    - Jan 09 17:09:35 EXEC ImportEchoMail
    + Jan 09 17:09:35 Process: Toss FDN/TIC Files
    + Jan 09 17:09:35 Waiting for BUSY nodes
    + Jan 09 17:09:35 Scanning Hatches
    + Jan 09 17:09:35 Hatching: MyGUI v1.0.2.26.zip from Mystic BBS Utils, Mods Etc.
    + Jan 09 17:09:35 Tossing to 21:1/100
    + Jan 09 17:09:35 Results: 0 import, 1 toss, 1 hatch, 0 bad in 0.02s


    The TIC file from Mystic contains

    Created By Mystic BBS v1.12 A47
    Area FSX_MUTL
    AreaDesc Mystic BBS Utils, Mods Etc.
    File MyGUI v1.0.2.26.zip
    Size 1669424
    CRC d39eeb54
    Replaces MyGUI v1.0.2.26.zip


    Htick at 21:1/100 (running Linux) tosses the file to other nodes...

    ----------------------- Sun 09 Jan 2022, htick/lnx 1.9.0-cur 2021-03-11
    1 Jan:09:2022:17:11:00 Start
    7 Jan:09:2022:17:11:00 Checking tmp dir
    C Jan:09:2022:17:11:00 Start tossing...
    7 Jan:09:2022:17:11:00 Processing Tic-File /hub/echomail/in/MyGUI v1.0.2.26.tic
    7 Jan:09:2022:17:11:00 File "MyGUI v1.0.2.26.zip": size: 1669424, area: FSX_MUTL, from: 21:1/101, orig: 21:1/101
    M Jan:09:2022:17:11:00 Moved /hub/echomail/in/MyGUI v1.0.2.26.zip to /ftn/files/fsx/fsx_mutl/MyGUI v1.0.2.26.zip
    M Jan:09:2022:17:11:00 Report file /ftn/files/announce/1cemsgdy.tic created for file MyGUI v1.0.2.26.zip
    S Jan:09:2022:17:11:00 Linked: /ftn/files/fsx/fsx_mutl/MyGUI v1.0.2.26.zip
    S Jan:09:2022:17:11:00 to: /hub/filebox/fsxnet_z21n2n100/MyGUI v1.0.2.26.zip


    Using BinkD (running Linux) as the mailer.. I noticed some Mystic nodes polling in were having this issue:

    + 09 Jan 19:19:38 [24867] call to 21:1/107@fsxnet
    09 Jan 19:19:38 [24867] trying bbs.thebytexchange.com [204.12.198.27]...
    09 Jan 19:19:39 [24867] connected
    + 09 Jan 19:19:39 [24867] outgoing session with bbs.thebytexchange.com:24554 [204.12.198.27]
    - 09 Jan 19:19:39 [24867] OPT CRAM-MD5-39f913bebc0dbe388a3671b63e62f6f8
    + 09 Jan 19:19:39 [24867] Remote requests MD mode
    - 09 Jan 19:19:39 [24867] SYS The ByteXchange BBS
    - 09 Jan 19:19:39 [24867] ZYZ Nugax
    - 09 Jan 19:19:39 [24867] TIME Sun, 09 Jan 2022 00:19:39 -0600
    - 09 Jan 19:19:39 [24867] VER Mystic/1.12A47 binkp/1.0
    - 09 Jan 19:19:39 [24867] BUILD 2020/10/13 12:55:27 Linux/64
    + 09 Jan 19:19:39 [24867] addr: 1:19/37@fidonet
    + 09 Jan 19:19:39 [24867] addr: 40:100/2@cybernet (n/a or busy)
    + 09 Jan 19:19:39 [24867] addr: 21:1/107@fsxnet
    + 09 Jan 19:19:39 [24867] addr: 10:102/1@araknet (n/a or busy)
    + 09 Jan 19:19:39 [24867] addr: 618:200/32@micronet (n/a or busy)
    + 09 Jan 19:19:39 [24867] addr: 42:256/6@sfnet (n/a or busy)
    + 09 Jan 19:19:39 [24867] addr: 700:100/12@spooknet (n/a or busy)
    + 09 Jan 19:19:39 [24867] addr: 42:972/1@dorenet (n/a or busy)
    + 09 Jan 19:19:39 [24867] addr: 11:1/200@wwivftn (n/a or busy)
    + 09 Jan 19:19:40 [24867] pwd protected session (MD5)
    + 09 Jan 19:19:40 [24867] sending /hub/filebox/fsxnet_z21n1n107/MyGUI v1.0.2.26.zip as MyGUI\x20v1.0.2.26.zip (1669424)
    - 09 Jan 19:19:40 [24867] QSIZE 0 files 0 bytes
    ? 09 Jan 19:19:40 [24867] M_GOT: cannot parse args
    + 09 Jan 19:19:40 [24867] done (to 21:1/107@fsxnet, failed, S/R: 0/0 (0/0 bytes))

    incoming nodes (earlier Mystic version) seem to have the same problem.

    + 09 Jan 23:58:08 [6668] incoming session with 47-220-170-115.gtwncmkt04.res.dyn.suddenlink.net [47.220.170.115]
    - 09 Jan 23:58:08 [6668] SYS Cold War Computing BBS
    - 09 Jan 23:58:08 [6668] ZYZ Jeff
    - 09 Jan 23:58:08 [6668] TIME Sun, 09 Jan 2022 04:58:08 -0600
    - 09 Jan 23:58:08 [6668] VER Mystic/1.12A46 binkp/1.0
    - 09 Jan 23:58:08 [6668] BUILD 2020/08/26 19:01:33 Raspberry Pi/32
    + 09 Jan 23:58:08 [6668] addr: 21:1/180@fsxnet
    + 09 Jan 23:58:08 [6668] pwd protected session (MD5)
    + 09 Jan 23:58:08 [6668] sending /hub/filebox/fsxnet_z21n1n180/MyGUI v1.0.2.26.zip as MyGUI\x20v1.0.2.26.zip (1669424)
    - 09 Jan 23:58:08 [6668] QSIZE 0 files 0 bytes
    ? 09 Jan 23:58:09 [6668] M_GOT: cannot parse args
    + 09 Jan 23:58:09 [6668] done (from 21:1/180@fsxnet, failed, S/R: 0/0 (0/0 bytes))

    My fix was to delete the file out of file boxes to get traffic flowing again.

    Hope that helps.

    Best, Paul

    --- Mystic BBS v1.12 A47 2021/11/06 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)
  • From g00r00@1:129/215 to Paul Hayton on Tuesday, January 11, 2022 11:42:39
    Using BinkD (running Linux) as the mailer.. I noticed some Mystic nodes polling in were having this issue:

    This seems to be related to a long standing issue with BinkD escaping filenames incorrectly. You'll have to get it fixed in BinkD.

    A47 has a workaround for the BinkD bugs but A46 would not.

    v1.0.2.26.zip as MyGUI\x20v1.0.2.26.zip (1669424)

    This is not a properly escaped filename being sent by BinkD as it does not follow the BINKP protocol specifications for filename escaping:

    5.2 File Name Issues
    --------------------
    In Mailer-parseable commands that contain a file name, the file name MUST NOT include a whitespace (ASCII value 20 hex). The file name SHOULD NOT include symbols other than alphanumeric (A-Z,a-z,0-9) and safe characters as defined below in BNF. All other symbols are to be considered unsafe and SHOULD be escaped in the form of two hexadecimal digits preceded by a backslash (e.g. a whitespace must be transmitted as "\20").

    filename = *pchar
    pchar = unreserved | escape
    unreserved = ALPHA | DIGIT | safe
    safe = "@" | "&" | "=" | "+" | "%" | "$" | "-" | "_" |
    "." | "!" | "(" | ")" | "#" | "|"
    escape = "\" HEX HEX

    ... Everyone smiles in the same language.

    --- Mystic BBS v1.12 A48 2022/01/10 (Windows/64)
    * Origin: Sector 7 * Mystic WHQ (1:129/215)
  • From Paul Hayton@3:770/100 to g00r00 on Wednesday, January 12, 2022 10:17:54
    On 11 Jan 2022 at 11:42a, g00r00 pondered and said...

    This seems to be related to a long standing issue with BinkD escaping filenames incorrectly. You'll have to get it fixed in BinkD.

    OK thanks for the info. I'll take this to BinkD dev.

    --- Mystic BBS v1.12 A47 2021/11/06 (Linux/64)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)
  • From g00r00@1:129/215 to Paul Hayton on Tuesday, January 11, 2022 16:27:49
    This seems to be related to a long standing issue with BinkD escaping filenames incorrectly. You'll have to get it fixed in BinkD.

    OK thanks for the info. I'll take this to BinkD dev.

    Great! Also in the meantime you can have any people using A46 upgrade to A47 which can work around that escape issue.

    Its probably pretty rare for spaces to be included though and certainly not in echomail packets so its probably a low priority in most cases.

    If you still have issues when everything is A47 please let me know there still could be something else going on (but that escape thing is an issue with BinkD regardless).

    ... A Mystic asking for a hot-dog: "Make me one with everything"
    --- Mystic BBS v1.12 A48 2022/01/11 (Windows/32)
    * Origin: Sector 7 * Mystic WHQ (1:129/215)