Select every .csv file at once (Ctrl/⌘-click to add to the selection) and drag them together onto the Trash.