Cook potatoes in boiling, salted, water for 20-25 minutes, or until potatoes are done but still firm. Drain potatoes in a colander and cool slightly.
In a large bowl, whisk together olive oil, red wine vinegar, mayonnaise, Greek seasoning and salt and pepper. Toss warm potatoes with dressing and stir in capers and olives. Serve immediately or chill in a covered dish to serve later.