Large Language Models (LLMs) are transforming the landscape of generative AI. Yet, their immense model sizes tether most LLMs to cloud environments, hindering their adaptability to specific downstream tasks and presenting challenges for scenarios involving private data. To address these issues, we propose a novel fine-tuning approach for end-to-end collaboration between a cloud-hosted LLM and an edge-based LLM agent, leveraging a Sandwiched Tuning framework. This approach not only boosts flexibility and scalability but also empowers users with heightened security and compliance, allowing tradeoffs between performance and cost. The proposed framework models cloud-edge collaboration as a nested optimization problem, which is under a grey-box constraint due to the cloud LLM’s parameters’ unavailability. Tailored to the unique problem structure, we introduce a computationally efficient nested Zeroth-order Cutting Plane (ZoCP) algorithm. We explore various collaboration modes, both parallel and serial, and conduct experiments to verify our effectiveness in each mode. Our extensive experiments reveal that our method delivers up to a 47.9% performance improvement over traditional methods. Additionally, we establish a convergence rate for ZoCP that is independent of the number of optimization parameters, highlighting its scalability on large-scale edge LLMs.