• Sunspear@piefed.social
    link
    fedilink
    English
    arrow-up
    12
    ·
    2 months ago

    Yeah I mean Wikipedia has regular dump files where you can just… download its entire content, or parts of it if you so wish. Getting money instead for that bandwidth is immediately an improvement

    • Nollij@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      2
      ·
      2 months ago

      They weren’t using those dumps. They were scraping the main site, at incredible expense to Wikipedia.