• File Size Issue

    From Barry Martin@454:1/1 to All on Friday, October 16, 2020 17:39:00

    Hi Folks!

    Putting the question here only because it involves *NIX systems.

    Issue is a file greater than 1 GB does not seem to transfer. Seems to
    be the or close to the 'magic number' cut off point: a 943.7 MB file transferred but a 1.4 GB did not.

    Source files are on a USB thumbdrive on a Raspberry Pi 4 running
    MotionEye OS -- surveillance utility with its own operating system. The
    files are (usually) immediately transmitted across the WiFi to the NAS.
    The NAS doesn't appear to have a problem with multi-GB files
    originating completely by wire (all Ethernet connection).

    I don't think there is a size limit with transferring via WiFi but is
    there a time-out limit? The transfer speed from the MotionEye RPi unit
    to my system (the computer I'm on currently) is rather slow, whereas transfering the same file after it has been transferred to the NAS is
    very quick.

    TIA!



    ¯ BarryMartin3@ ®
    ¯ @MyMetronet.NET ®

    ... Bad pet gifts: Centipede: 50 pairs of 5-inch heel open toe pumps
    --- MultiMail/Win32 v0.47
    þ wcECHO 4.2 ÷ ILink: The Safe BBS þ Bettendorf, IA

    --- QScan/PCB v1.20a / 01-0462
    * Origin: ILink: CFBBS | cfbbs.no-ip.com | 856-933-7096 (454:1/1)
  • From Ky Moffet@454:1/1 to Barry Martin on Saturday, October 17, 2020 18:14:00
    BARRY MARTIN wrote:
    Hi Folks!

    Putting the question here only because it involves *NIX systems.

    Issue is a file greater than 1 GB does not seem to transfer. Seems to
    be the or close to the 'magic number' cut off point: a 943.7 MB file transferred but a 1.4 GB did not.

    Source files are on a USB thumbdrive on a Raspberry Pi 4 running
    MotionEye OS -- surveillance utility with its own operating system. The files are (usually) immediately transmitted across the WiFi to the NAS.
    The NAS doesn't appear to have a problem with multi-GB files
    originating completely by wire (all Ethernet connection).

    I don't think there is a size limit with transferring via WiFi but is
    there a time-out limit? The transfer speed from the MotionEye RPi unit
    to my system (the computer I'm on currently) is rather slow, whereas transfering the same file after it has been transferred to the NAS is
    very quick.

    If there's a limit, I'd guess it's MotionEye that's failing, not *NIX anything.

    Timeout should not come into play unless no data is transferred for
    however long, not just because the file is big and takes a Long Time.

    I'd be suspicious that maybe MotionEye caches the whole file in swap (or worse, in RAM) before sending it off, and when it runs out of cache
    space, that's when it fails.
    þ RNET 2.10U: ILink: Techware BBS þ Hollywood, Ca þ www.techware2k.com

    --- QScan/PCB v1.20a / 01-0462
    * Origin: ILink: CFBBS | cfbbs.no-ip.com | 856-933-7096 (454:1/1)
  • From Barry Martin@454:1/1 to Ky Moffet on Sunday, October 18, 2020 10:31:00

    Hi Ky!

    BARRY MARTIN wrote:
    Hi Folks!

    Putting the question here only because it involves *NIX systems.

    Issue is a file greater than 1 GB does not seem to transfer. Seems to
    be the or close to the 'magic number' cut off point: a 943.7 MB file transferred but a 1.4 GB did not.

    Source files are on a USB thumbdrive on a Raspberry Pi 4 running
    MotionEye OS -- surveillance utility with its own operating system. The files are (usually) immediately transmitted across the WiFi to the NAS.
    The NAS doesn't appear to have a problem with multi-GB files
    originating completely by wire (all Ethernet connection).

    I don't think there is a size limit with transferring via WiFi but is
    there a time-out limit? The transfer speed from the MotionEye RPi unit
    to my system (the computer I'm on currently) is rather slow, whereas transfering the same file after it has been transferred to the NAS is
    very quick.

    If there's a limit, I'd guess it's MotionEye that's failing, not
    *NIX anything.

    That's what I was thinking.


    Timeout should not come into play unless no data is transferred
    for however long, not just because the file is big and takes a
    Long Time.

    Based on other experiences agree. As long as the connection is active
    and something else isn't imposing a limitation then the connection
    should be maintained. Didn't think there was a 'rule' but figured I'd
    ask anyway -- gotta learn!


    I'd be suspicious that maybe MotionEye caches the whole file in
    swap (or worse, in RAM) before sending it off, and when it runs
    out of cache space, that's when it fails.

    Possible; no idea and may not be able to find out as can't get to that
    part, or at least I haven't figured out how. (The older Raspberry Pi
    utility for duplicating [micro]SD cards wouldn't copy the partition
    MotionEye used for OS.)

    And yesteday I had to reboot my WiFi access point: the connected devices
    were and weren't working. Could ping some but they stopped getting
    data. Not sure if that had something to do with the file size problem
    as go GB files to transfer since I posted. And based on previous
    versions I can't upload/insert a file to test.



    ¯ BarryMartin3@ ®
    ¯ @MyMetronet.NET ®

    ... There is only one smartest dog in the world, and every kid has it.
    --- MultiMail/Win32 v0.47
    þ wcECHO 4.2 ÷ ILink: The Safe BBS þ Bettendorf, IA

    --- QScan/PCB v1.20a / 01-0462
    * Origin: ILink: CFBBS | cfbbs.no-ip.com | 856-933-7096 (454:1/1)
  • From Ky Moffet@454:1/1 to Barry Martin on Sunday, October 25, 2020 18:18:00
    Here's a thread with complaints about slow after a certain size, I
    wonder if it's related.

    https://www.pclinuxos.com/forum/index.php/topic,153868.msg1319654.html
    þ RNET 2.10U: ILink: Techware BBS þ Hollywood, Ca þ www.techware2k.com

    --- QScan/PCB v1.20a / 01-0462
    * Origin: ILink: CFBBS | cfbbs.no-ip.com | 856-933-7096 (454:1/1)
  • From Barry Martin@454:1/1 to Ky Moffet on Monday, October 26, 2020 06:53:00

    Ki Ky!

    Here's a thread with complaints about slow after a certain size,
    I wonder if it's related. https://www.pclinuxos.com/forum/index.php/topic,153868.msg1319654.
    html

    Thanks! The issue was more not copying files greater than 1 GB but I
    have also noticed the slowdown in copying on varios machines so good information to check.

    As for the > 1 GB issue, I need to go back in and check -- busy, plus I cheated a little and put together a script to copy the files that hadn't
    been yransferred originally. Also found there may have been an issue
    with my WiFi AP as it started dropping connections.


    ¯ BarryMartin3@ ®
    ¯ @MyMetronet.NET ®

    ... Plumber's Motto: "Clogs Aren't Just for the Dutch"
    --- MultiMail/Win32 v0.47
    þ wcECHO 4.2 ÷ ILink: The Safe BBS þ Bettendorf, IA

    --- QScan/PCB v1.20a / 01-0462
    * Origin: ILink: CFBBS | cfbbs.no-ip.com | 856-933-7096 (454:1/1)